A superb last-gasp goal from substitute Mickey Evans earned West Brom boss Brian Little his first Hawthorns league success of the season.
The Republic of Ireland striker joined the action with just nine minutes left on the clock and found the target with a cracking right-footer to end Albion's run of six draws and two defeats in front of their home supporters.
But Albion, booed off the pitch at the break when they were losing 2-1 to Pompey, had to pull out all the stops to bank maximum home points.
It was Little's men who opened the scoring when Italian starlet Enzo Maresca found the net from the narrowest of angles in the 14th minute after dribbling his way past keeper Aaron Flahavan.
But it was then the turn of Alan Ball's Pompey to stun Albion with two goals in nine minutes.
Veteran striker Guy Whittingham levelled the score with a marvellous glancing header in the 20th minute after a right-wing cross from Michael Panopoulos.
Then, in the 29th minute, skipper Andy Awford found the net to give Pompey a deserved lead.
Fitzroy Simpson chipped in a perfect corner with Awford powering in a fine header that was scooped away by defender Andy McDermott.
But Leicester referee Paul Danson was perfectly positioned and he ruled that the ball had first crossed the line before being booted clear.
After the break it was Albion who took the initiative and it was 2-2 in the 59th minute when defender Jason van Blerk grabbed his first goal in two-and-a-half years at the Hawthorns.
The Aussie defender made no mistake with a powerful right-footer after the ball had been laid back by striker Lee Hughes.
Pompey were then reduced to 10 men in the 67th minute when Simpson was given his marching orders for his second bookable offence - just 60 seconds after a first yellow card flashed for time-wasting.
Albion continued to press and they had their fans cheering when Evans made it 3-2 with his 89th minute shot after the ball had been slotted into his path by James Quinn.