Los Blancos' title bid looked to be wavering when they fell two goals behind inside 14 minutes, but a brace from Gareth Bale and a Lucas Vasquez header saw Zinedine Zidane's side record all the points.
Real Madrid were without the services of Cristiano Ronaldo, who missed out due to a minor thigh problem, but Bale was fit to return from a back injury and started alongside Jese Rodriguez and Karim Benzema.
Real Madrid came close to taking the lead in the third minute when Bale struck one from the edge of the box, but the Welshman's low effort rattled the post before dropping wide of Juan Carlos's goal.
It was the home side that made the breakthrough in the seventh minute, however, when Bebe drove down the left before delivering a low cross in to Adri Embarba, who had the simple task of firing home from close range.
Benzema, who was forced off with an injury late in the first period, had a half-chance in the 12th minute when Kovacic picked out the Frenchman with a pass, but the striker's overhead kick was always missing the target.
Rayo had obviously not read the pre-match script though and doubled their lead in the 14th minute when Miku smashed the ball home from close range after Real Madrid had failed to defend a corner from the right.
The visitors were all over the place at the back in the opening 25 minutes and only smart goalkeeping from Keylor Navas prevented Embarba from sending Rayo into a three-goal lead in the capital. Bebe also sent a swerving effort towards the top corner just before the 30-minute mark, only for Navas to bail out his team once again.
Real Madrid improved as the first period approached its latter stages, however, and were back in the match in the 35th minute when the influential Bale headed a Toni Kroos corner past Carlos to silence the home supporters.
Embarba, some three minutes later, should have scored his second of the match when he found himself unmarked at the far post, but the Spaniard somehow headed over the crossbar as it remained a one-goal game.
Both teams continued to commit players forward as the half-time whistle approached and Bale sent a curling effort just wide of the Rayo post as Real Madrid entered the break behind in the Spanish capital.
The visitors came out firing in the second period and levelled the scores in the 52nd minute when Vasquez headed a Danilo cross into the back of the net.
Vasquez had a wonderful chance to complete the turnaround just before the hour when Bale delivered a low cross into the Rayo box, but the Spaniard just could not make contact, before Jese was denied by the palms of Carlos.
With 65 minutes on the clock, it was Real Madrid that looked the most likely to grab a third goal, and Luka Modric's introduction in the middle of the park further increased the away side's control of events in the capital.
Roberto Trashorras curled a long-range strike into the arms of Navas as Rayo registered a rare attempt on target in the second period, but it was the visitors that continued to search for a winning goal.
James Rodriguez was thrown on just before Bale hit the wall from a free kick just outside the Rayo box, but Bale finally completed the turnaround in the 81st minute when he drove away from the home side's defence before finding the bottom corner with a low strike.
The victory has sent Real Madrid two points clear at the top of the table, but they would find themselves back in third if Atletico and Barcelona beat Malaga and Sporting Gijon respectively on Saturday night.