The capital outfit are regarded as one of the most entertaining teams in La Liga, but they have conceded 51 times in their 21 matches thus far, which is the worst defensive record in the division.
"After the break we did very well. But when you give a team like Atleti an advantage, it's tough," he told reporters at a press conference.
"But my players left their hearts on the pitch looking for a third and fourth goal. We have such a clear disadvantage, so to compete with them is heartening.
"The crowd can react however they want – it is not the players who responsible, it is the coach telling them to play that way. The jeers will not change the way this team plays football."
Rayo currently sit 19th in La Liga.