After Jean Beausejour swept Latics into the lead inside 20 minutes, missed chances were looking to be the route of a draw after Davide Santon's equaliser with 18 minutes remaining but Arouna Kone popped up in the box to poke home a 91st minute scramble.
Roberto Martinez started with the same team as the FA Cup quarter final victory over Everton last weekend, with Callum McManaman and Joel Robles getting their first ever Premier League starts.
It was a high octane first half in spells but there was little incident until Jean Beausejour opened the scoring on 18 minutes.
McManaman, the man who brought about the games' biggest talking point for a high challenge on Massadio Haidara, sold Newcastle's full back with a drop of the shoulder down the right and curled a cross to the back post, a poor touch back by the defender at the back post fell to Beausejour who just swept the ball under Rob Elliot's body.
Four minutes later came McManaman's tackle which ended the game for Haidara as he caught the defender over the top of the ball.
On 35 minutes Latics could have made it 2-0 when Shaun Maloney cut in from the left and curled one towards the bottom corner, only for Elliot to save low to his left and push the ball just out of Boyce's way at the back post.
Three minutes before half time James McCarthy then broke through the middle with Beausejour's pass cutting Newcastle's defence in two but the Ireland international was chased down by Jonas Gutierrez who blocked McCarthy shot at the last second.
In the second half the game opened up and made it end-to-end but the final product wasn't there for either side.
Just before Santon's equaliser James McArthur called for a penalty as his right footed shot from the edge of the area hit Steven Taylor's arm and went wide for a corner.
Then came the equaliser, a simple but effective goal which was played from the left touch line to the edge of the area on the right hand side.
Santon controlled well in space and was stood up by Figueroa but a drop of the shoulder caught him out and Santon slotted the ball under Joel Robles' body.
With over 15 minutes to go Latics had more chances on the break but couldn't muster a good enough shot.
In the first of four added minutes Latics snatched the points when a corner from Jordi Gomez seemed to come off the arm of Maynor Figueroa and bounced towards Paul Scharner on the edge of the six yard box.
The scramble ensued and an attempted clearance came off Franco Di Santo into the path of Kone who hooked his leg around the ball from close range to send the DW Stadium into madness.
Latics saw out the three points and could have scored a third when Rob Elliot raced out of his goal but had it taken off his toe by Kone who entered a foot race towards an open goal with Vurnon Anita but was just edged out.
The whistle blew and the DW erupted as Latics sealed a vital three points.