The home side looked like they were going to condemn Michael O'Neill's team to defeat in their opening Euro 2016 qualifer when Tamas Priskin headed home from close range with 15 minutes remaining.
However, after Kyle Lafferty had set up Niall McGinn to tap home from three yards, the duo switched roles as Lafferty bundled the ball into the net to give Northern Ireland their first away victory in four years.
Read below to see how the action unfolded in Budapest.
4.30pmHello, and welcome to Sports Mole's live coverage of the Euro 2016 qualifier between Hungary and Northern Ireland.
4.33pmIt's been just seven weeks since the end of the World Cup finals in Brazil but the qualification campaign for the next major tournament - Euro 2016 in France - is ready to get underway. From Northern Ireland's perspective, they will be hoping for a much improved year after winning just one match when they attempted to qualify for Brazil.
4.37pmAs for Northern Ireland boss Michael O'Neill, he has accepted that his job could be on the line after winning just one match - at home to Russia - in 18 fixtures in charge of his country. Despite the mediocre record, he will also believe that a win in Budapest will create some much needed optimism for their upcoming games as they attempt to finish in the top three of a six-team group.
4.38pmRight then, with just over 20 minutes to go until kickoff, let's crack on with the team news.
4.48pmAs for Hungary, Bolton Wanderers goalkeeper Adam Bogdan misses out to Peter Gulacsi but Watford midfielder Daniel Tozser gets a start in midfield. Dynamo Moscow midfielder Balazs Dzsudzsak will captain his side in Budapest.
4.52pmPREDICTION! With Hungary having narrowly missed out on earning a place in the playoffs in their previous two qualification campaigns, this is likely to prove to be one of Northern Ireland's toughest games of the group. There will be a cautiousness about Michael O'Neill's side, but if they fall behind against their hosts, they could find themselves struggling to contain them throughout the 90 minutes. 2-0 to Hungary.
4.56pmThe two teams have made their way out onto the pitch at the Groupama Arena. The standard fair-play and anti-racism messages are being read out to the crowd before we observe the respective national anthems.
3 minNorthern Ireland have had plenty of the ball in the opening few minutes. Most of their possession is in the middle third of the pitch but it's an encouraging start.
4 minHungary have wasted no time in making their presence known against Kyle Lafferty. The Norwich City striker has been known to lose his cool on occasions, but hopefully he can keep his composure this afternoon.
5 minCHANCES! Both teams create their first opportunities of the match within 30 seconds of each other. After Jamie Ward has scuffed a chance on the edge of the Hungary area, the home side went up the other end with Balazs Dzsudzsak slicing the ball wide of the far post from 12 yards. Decent chance that.
8 minNorthern Ireland have look very lively in these first eight minutes and after the ball broke kindly for Conor McLaughlin down the right-hand side, his cross found Lafferty who could only loop his head onto the roof of the net.
12 minSince that chance, Hungary had managed to establish control of possession but they don't go anywhere fast and the ball is eventually kicked out for a goal kick. For the third time, a Hungary defender - Vilmos Vanczak - goes through the back of Kyle Lafferty but the striker is quickly back up on his feet.
16 minHungary win a corner after good fine work down the right flank, but Northern Ireland just about survive after Daniel Tozser's volley from the edge of the penalty area caused mayhem in front of Roy Carroll's goal.
19 minThere's a feeling that Hungary are under pressure to perform in front of their own fans, and it's playing into Northern Ireland's hands. Jamie Ward and Conor McLaughlin are looking particularly threatening down either flank and the visitors have a chance to gain control of the match here.
23 minAfter saying that, Hungary are back on the offensive and Northern Ireland are having to form a collective rearguard to ensure that they don't concede against the hosts. When they do regain the ball, they look a threat on the break but on this occasion, Conor McLaughlin's burst down the right was brought to an abrupt halt by a Hungary defender.
26 minGergely Rudolf and Balazs Dzsudzsak are beginning to pull the strings in the final third for Hungary. The home side earn a succession of corners and set pieces which is forcing Northern Ireland onto the back foot but for now, they survive.
30 minWe have reached the half-hour mark in Budapest and we are still goalless. Northern Ireland have more than held their own against a team ranked over 50 places higher than them in the world rankings, but the home side are beginning to improve after a slow start.
33 minMore pressure from Hungary results in Balazs Dzsudzsak retrieving the ball on the right side of the penalty but after switching the ball onto his left foot to cross into the middle, Roy Carroll comes out to bravely punch clear.
35 minBOOKING! Vilmos Vanczak goes into the book after cynically pulling down Kyle Lafferty before he entered the penalty area. The striker's turn deceived the defender and he had little choice but to haul him down.
39 minBalazs Dzsudzsak is starting to antagonise some of the Northern Ireland players after repeatedly throwing himself to the ground at every opportunity. He is showing plenty of promise on the ball but each time a challenge comes in, he is theatrically falling over.
42 minA few boos and whistles come from the home crowd after another stray pass goes out for a throw-in. I think that they were expecting to be ahead by now, but their team are being met by a fine display from Michael O'Neill's men.
44 minBOOKING! Olivier Norwood has had his name taken by the referee after sliding in late on Jozsef Varga. Deserved yellow.
45 minCHANCE! Northern Ireland miss a huge chance to take the lead just before half time. After he closed down the ball on the halfway line, Jamie Ward ran clear on goal but after his shot was saved by Peter Gulasci, Corry Evans directed the rebound over the crossbar. It was a difficult chance but one that he should be taking.
5.50pmThe half-time whistle has been blown in Budapest and Northern Ireland have managed to hold Hungary at the break. Hungary look a threat in and around the box but they haven't forced Roy Carroll into a significant save, but Northern Ireland have had their moments too, with Jamie Ward and Corry Evans squandering superb chances just before the end of the first 45 minutes.
5.56pmHungary have created six attempts compared to Northern Ireland's five, but it's the home side who are winning the corner count by six to one. There has been 19 fouls in the first half, but it hasn't really been a dirty game.
6.00pmRepublic of Ireland are also in action this afternoon, with Martin O'Neill's side currently being held 1-1 in Georgia. If you wish to follow that match as well as this one, you can do so by clicking here.
46 minWe are back underway in the second half. Hungary have introduced Tamas Priskin into the action.
48 minIt's been all Hungary at the start of this second half but they haven't been able to get an early shot in on Roy Carroll's goal. Northern Ireland need to keep their concentration to ensure that their first-half performance doesn't go to waste.
51 minNorthern Ireland win a corner after good clever work down the left from Chris Brunt but despite the ball into the box being a threatening one, Gareth McAuley gives away a free kick at the back post.
55 minHungary are still having much of the ball, but so far, Northern Ireland are doing an effective job in stifling their efforts to open the scoring. Michael O'Neill will have to decide when to make his first change because his team have put in a lot of effort so far.
58 minThe referee in this match has been extremely whistle-happy and it's hardly helped the flow of the match. He gives the impression that he has a decision in him that could effect the result of this match, but let's hope I'm wrong.
61 minJamie Ward has had a tremendous game for Northern Ireland and he majestic pulls the ball out of the air before flicking it over the top of the Hungary defence. However, it has just too much pace on it and it eventually trickles away from Kyle Lafferty.
63 minSAVE! Roy Carroll hasn't had a whole lot to do in this match but he had to be at his best to keep the score level when Gergely Rudolf let fly from 20 yards, forcing the former Manchester United stopper to tip the ball away from a corner.
66 minSUBSTITUTION! Moments after mentioning that Jamie Ward had been excellent for Northern Ireland, he is forced off with what appears to be a muscle injury. He has been replaced by Niall McGinn.
69 minCorry Evans has a glimmer of an opportunity after being found at the back post but after controlling the ball well, he fired his resulting volley at the defender and the ball was cleared.
70 minSUBSTITUTION! Gergely Rudolf has been, by far, Hungary's best player but he has been hauled off for Istvan Kovacs.
72 minSUBSTITUTION! Gareth McAuley has been moving rather gingerly over the past few minutes and he has been replaced by Craig Cathcart. We have 18 minutes to go in Budapest. Can Northern Ireland hang on?
75 minUnfortunately, the answer to that question is no. Balazs Dzsudzsak swung in the corner from the right and Tamas Priskin rose above the Northern Ireland defence to power a header past Roy Carroll from six yards. It's tough on Northern Ireland but Hungary had been building up a head of steam.
79 minSUBSTITUTION! The last throw of the dice for Michael O'Neill comes in the form of Billy McKay, who has been thrown on in the place of Olivier Norwood.
81 minHow about that then! Northern Ireland are level in Budapest and it's a terrifically worked goal. After a quick one-two on the edge of the penalty area, Kyle Lafferty powered into the penalty area before pulling the ball across goal for Niall McGinn to tap home from three yards.
83 minIt's going from bad to worse for Hungary. Zoltan Liptak does well to stop a break-away down the left, but in doing so, he has pulled his hamstring. However, with Hungary having used their three substitutions, they will end this match with 10 men.
87 minDespite being a man down, Hungary are making most of the running in the closing stages, but Northern Ireland look a threat on the break. There's still another goal in this match - neither side are sitting on the draw.
88 minUnbelievable scenes in Budapest! Northern Ireland take the lead and it's another wonderful goal. After the ball found Niall McGinn on the right-hand side of the penalty area, he drove the ball across the penalty area for Kyle Lafferty to bundle the ball into the net, despite the attentions of a Hungary defender.
90 minChris Baird tries his luck from 30 yards but his shot was never threatening a third goal. There will be four minutes of added-on time.
90+2 minHungary are starting to press forward, as you would expect, but so far, Northern Ireland are holding firm. Two minutes to go.
6.52pmThey've done it! What a result for Northern Ireland and Michael O'Neill. They have gone to Hungary, one of the favoured teams in the group, and come away with all three points in such dramatic circumstances. They fully deserve it as well.
6.53pmThat will be all from Sports Mole for this match but we hope you stick around throughout the evening - we have got Germany against Scotland coming up at 7.15pm. Goodbye for now.