The Spaniard recently criticised the club's owners over their decision to assign him the job title of 'interim manager', and the ex-Sky Sports pundit empathises with his frustration.
"He's been put in an impossible situation," Gray told talkSPORT. "As soon as you announce him as interim manager, he's right, his position is almost untenable and weak because the players know he's not going to be there.
"[Appointing] Rafa was a massive mistake from Roman Abramovich. As soon as he was put in there as interim manager, they gave themselves a massive problem. How do you motivate these players if they don't want to be motivated?"
Benitez also lashed out at the club's supporters calling for his head, claiming that they should be getting behind their team, rather than "wasting time making banners".
Gray added: "Some of the fans have been over the top. I don't mind fans moaning, having a say on their team, prior to the game or after the game.
"That's fine. But surely the supporters, even if they detest Rafa, which a lot of them seem to do, should focus on supporting their team in that 90 minutes.
"I think it was a mistake from day one. I think he should walk this morning."
Benitez recently confirmed that he will see out his Chelsea contract and remain at the club until the end of the season.