Jonny Evans's header looked to have put United through to a showdown with Manchester City, but with just a minute of extra time remaining, goalkeeper David de Gea spilled a Phil Bardsley shot into the bottom corner.
However, 60 seconds later, Javier Hernandez struck to force a penalty shootout, but after just three penalties had been scored in the first nine attempts, Rafael saw his kick saved by Vito Mannone to give the Black Cats a place in the final.
Below, Sports Mole takes a look back at a dramatic game at Old Trafford.
On target: 6
On target: 1
Was the result fair?
Manchester United created the better chances throughout the 120 minutes, but they can have no complaints with the result. Sunderland's play was on par with that of their hosts, and while their goal came somewhat fortuitously, they deserved that bit of luck for the amount of work that they put in at Old Trafford. Gus Poyet has worked wonders with the team spirit at the Stadium of Light.
Manchester United's performance
Despite Javier Hernandez forcing a fine stop out of Vito Mannone and Darren Fletcher striking a post, United were disappointing until they netted their opener. Adnan Januzaj was a frequent threat for David Moyes's side, but they lacked a cutting edge in attack, and they were made to pay for it. It wasn't the worst display that United have put in since the turn of the year, but it's not one that the United supporters will want to witness every week.
The visitors were no more than solid for the opening 45 minutes, but they were doing a job to remain in the tie. However, without testing David de Gea, they had the better of the second half with United allowing them plenty of time on the ball. They pressed for the majority of the extra period, and they deserved their goal when it came. United's second goal was a huge sucker-punch, but even though they only scored two out of five penalties, they held their nerve better than their opponents.
Sports Mole's man of the match
Vito Mannone: If Manchester United had won this match, Adnan Januzaj would have won this accolade, but Vito Mannone's heroics in the penalty shootout swung it in the Italian's favour. The former Arsenal stopper was solid throughout the opening 120 minutes, but it was his save from Januzaj that changed the course of the shootout. He then capped off a thrilling night by getting in the way of Rafael's effort to earn his side the win.
Biggest gaffe There were seven contenders for this in what is likely to be the worst penalty shootout that you will ever see! However, it's got to go to David de Gea. United would be at Wembley if the Spaniard hadn't helped Phil Bardsley's effort into the net from what was Sunderland's only shot on target in 120 minutes.
Referee performance Lee Mason got the odd decision wrong, including the awarding of a goal kick when Sunderland should have had a corner during a dominant spell of play, but the official had a solid outing and he had no direct effect on the result of the game.
Manchester United: United's early exit from the FA Cup means that they will have to wait until next Tuesday to try to offer a response to tonight's disappointment. They will host Cardiff City at Old Trafford.
Sunderland: Sunderland will receive a rapturous reception when they host Kidderminster Harriers in the FA Cup on Saturday.