Los Blancos were leading 3-0 after 50 minutes, but two penalties in as many minutes offered the away side a lifeline.
Despite clinging on to secure all three points, Ancelotti was left fuming at the ease with which his side offered Rayo a route back into the game.
"We need to change," Ancelotti told a post-match press conference. "We have to lose our superficiality. The second half was a disaster.
"Against Sevilla, we were leading 3-0, but we allowed them to get back into contention. The same thing happened tonight and it's not by coincidence. We need to change and lose the superficiality we're showing out on the pitch. You cannot let the other team back into a game you are winning when you have the quality Madrid has.
"I will talk to the players in private, not here. I'm not happy with what happened. We should be able to read the game better and hit our opponents on the break. The second half was disastrous."
Real Madrid are currently in third place in La Liga, six points adrift of leaders Barcelona.