Madrid had a whole host of chances to find the back of the net in the Spanish capital, but Villarreal netted three minutes from time to pile more misery on Zinedine Zidane's side, who stay 16 points off La Liga leaders Barcelona.
Both teams had their moments in what was a tight opening 10 minutes, with Nacho seeing a goal-bound strike blocked for Real Madrid, while Samu Castillejo was threatening down the other end of the field.
Gareth Bale had the ball in the back of the Villarreal net in the 13th minute, but the Welshman's header was ruled offside, with replays confirming that it was indeed the correct decision.
Marcelo then saw his powerful strike saved by Sergio Asenjo in the 19th minute as Real Madrid started to force the issue. Cristiano Ronaldo was next to come close four minutes later as he smashed the crossbar with a dipping free kick.
Ronaldo had another super chance just before the 30-minute mark when he broke into the Villarreal box, but the number seven lifted the ball just wide of the far post, before smashing into the side-netting from a similar area moments later.
Real Madrid had two more excellent chances late in the first period, but Bale curled just wide of the far post before Ronaldo's close-range effort was somehow kept out by the diving Asenjo.
Denis Cheryshev, who came off the Villarreal bench at the interval, had a glorious chance to send his team ahead early in the second period, but the Russian missed the target from just outside the Real Madrid box.
Kroos stung the palms of Asenjo in the 52nd minute as Real Madrid pushed forward, with Ronaldo, Bale and Isco all heavily involved in the action.
Villarreal were still a serious danger, however, and Bacca smashed the ball into the side-netting just past the hour after brilliant work from Castillejo in a wide position.
Modric fired high of the Villarreal crossbar in the 80th minute as another good chance came and went for the home side.
It did appear that the points would be shared at the Bernabeu, but Villarreal incredibly took the lead in the 87th minute when Fornals clipped over Navas after the Real Madrid goalkeeper got down low to keep out a strike from Cheryshev.
It turned out to be quite a comfortable conclusion for Villarreal as Real Madrid's stuttering league form, which has effectively ended their chances of catching Barcelona, continued in front of their own supporters.
REAL MADRID (4-3-3): Navas; Carvajal, Varane, Nacho, Marcelo; Casemiro, Modric, Kroos; Isco (Lucas 70'), Ronaldo, Bale (Asensio 70')
VILLARREAL (4-1-2-1-2): Asenjo; Mario, Bonera, Alvaro, Costa; Rodrigo, Castillejo (Rukavina 84'), Trigueros, Fornals; Raba (Cheryshev 46'), Bacca (Unal 76')