Los Blancos entered the match holding a 1-0 lead from the first leg of their last-eight tie, but it was the visitors that progressed on another hugely disappointing night for the Spanish champions.
Sergio Ramos was back for Real Madrid - who have now lost three of their last five home games in all competitions - after recovering from injury, while Karim Benzema started his first match since before Christmas.
As for Leganes, Nabil El Zhar supported on-loan Celta Vigo forward Claudio Beauvue in the final third, but there were changes from the team's 2-2 draw against Alaves at the weekend, with Nordin Amrabat among those to come in.
Leganes came close to taking the lead in the fifth minute when Beauvue sent a clever free kick towards the bottom corner, but the number 12 saw his effort hit the post.
Leganes were comfortably the better team in the first 25 minutes of football, and the game's second attempt on target also came for the away side - Gabriel testing Casilla with a header from a corner.
Real Madrid just could not get going in front of their own supporters, and Leganes took the lead in the 31st minute when Javier Eraso found the top corner with a stunning strike following a loose pass from Achraf Hakimi.
Benzema had a super chance to level the scores in the 38th minute, but the Frenchman just failed to make contact on a cross from Isco as the tie stayed 1-1 at the end of a very interesting first half of football.
Real Madrid levelled the scores in the 47th minute, however, when Vazquez's clever pass was chipped into the back of the net by Benzema from a narrow angle.
Leganes refused to be dispirited though, and the away side scored a second in the 55th minute when Gabriel headed a corner from Eraso into the back of the net.
Dani Carvajal and Luka Modric both came off the Real Madrid bench with little over 20 minutes remaining, but it was still Leganes having the better chances, with Eraso seeing his goal-bound effort blocked by Theo Hernandez.
Borja Mayoral was the third player off the home side's bench in the 77th minute, and the pressure increased on the Leganes goal as Los Blancos looked to avoid a quarter-final elimination.
Benzema had a stunning chance to level the scores in the 81st minute when he met a low cross from Modric, but Nereo Champagne was on hand to make an excellent save, before Mayoral headed wide moments later.
Champagne kept out a looping Ramos header in the 85th minute as the pressure continued into the latter stages, but Leganes held on to record a famous Bernabeu win - booking their semi-final spot in the process.
REAL MADRID (4-3-3): Casilla; Achraf (Carvajal 68'), Ramos, Nacho, Theo; Kovacic, Llorente (Modric 68'), Isco (Mayoral 77'); Lucas, Benzema, Asensio
LEGANES (4-2-3-1): Champagne; Tito, Bustinza, Siovas (Mantovani 58'), Rico; Gabriel, Brasanac; El Zhar (Naranjo 80'), Eraso, Amrabat (Garcia 70'); Beauvue