The Gunners trailed to a Sergio Aguero goal heading into the final quarter of the game, but Nacho Monreal fired home to take the tight semi-final the distance and Sanchez proved the hero in the 101st minute.
City hit the frame of the goal twice in the latter stages of the contest and had a goal incorrectly ruled out in the first half, but for the first time in nine appearances at this stage of the competition they came out second best in the end.
It was the officials who were more involved in the action in a goalless first half, being made to run the rule over a number of contentious decisions.
Both sides managed just one shot on target in the opening 45 minutes, coming in the space of 60 seconds as Olivier Giroud aimed a header straight at Claudio Bravo at one end and Silva was denied by Petr Cech at the other - his last involvement in the match.
Aguero blasted into the side-netting from his first real sight of goal, but he felt that he should have been awarded a penalty on the half-hour mark when his heels were clipped by Alex Oxlade-Chamberlain.
Arsenal had a strong shout of their own turned down in the dying embers of the half, with Jesus Navas focusing more on dragging Alexis Sanchez to the ground than winning the ball, yet it was City who were more aggrieved to go into the break all square.
The Citizens saw far more of the ball in dangerous areas against an Arsenal side continuing its three-at-the-back experiment, and they should have been rewarded through substitute Raheem Sterling's close-range finish.
Leroy Sane's cross was turned goalwards by Aguero at the back post, only for Cech to parry into the path of Sterling who was waiting to pounce a few yards out, but the officials felt that Sane's initial cross had gone out of play - replays suggesting otherwise.
These sides had netted 207 goals between them in all competitions this season, but clear-cut chances were proving tough to come by at either end.
Nicolas Otamendi nodded into the ground and wide from a corner early in the second period and Sanchez was so nearly in following Bravo's slip, but the breakthrough moment did eventually arrive shortly after the hour.
Arsenal gave away possession through Aaron Ramsey and, a Yaya Toure ball over the top later, Aguero was clean through on goal and dinking it over Cech after his initial first touch tempted the Czech to race out.
A 12th goal in 12 outings for Aguero gave City, who lead the Gunners by seven points in the Premier League after playing one game more, something to hold on to in the remaining 30 minutes.
Hold on they could not, though, as the Gunners - competing at this stage for a 29th time - levelled up soon after when their two wing-backs combined to justify Arsene Wenger's big gamble.
Oxlade-Chamberlain swung in a delightful cross from the right, which was met emphatically by unlikely scorer Monreal at the back post with a right-footed volley past Bravo.
There was a late chance for Toure to make himself a Wembley hero once again, sending a volley towards the bottom corner which Cech turned onto the post with his fingertips, while Fernandinho was denied moments later with a thumping header against the bar.
As the minutes ticked towards extra time, substitute Danny Welbeck decided against picking out Ramsey and instead attempted to whip the ball into the far corner, missing by a matter of inches in a thrilling end-to-end conclusion.
Rob Holding missed a good chance to edge his side ahead for the first time six minutes into extra time, but the Gunners did not have to wait much longer for the winning moment.
Sanchez bagged his fourth goal in three appearances at Wembley, prodding home when Ozil's free kick into the box fell into his path to send Arsenal through to an FA Cup final meeting with London rivals Chelsea in five weeks' time.