The England number one fell out of favour with the City boss earlier this season after a handful of errors, particularly away from the Etihad Stadium, which cost the 2012 champions points in the Premier League.
However, Cascarino, now an analyst for The Times, has insisted that Hart left Pellegrini with little choice but to remove him from the starting lineup.
"If you are looking for a scapegoat, it's not very nice, but Joe Hart has made mistakes in a lot of the away games. Whether it be Cardiff [City], [Aston] Villa or at Chelsea, he made big errors that have cost them," he said.
"If they stop making mistakes from the goalkeeper, which has happened with Hart, they will be fine and will win lots of football matches. If you are playing for a top side and are a striker that misses lots of chances, you'll get dropped.
"It's got to be the same with a goalkeeper, you can't keep your place. You can't continue with a goalkeeper that is making vital errors in big games that sees you lose points. A manager can't do that and Pellegrini has made totally the right decision."
Hart has sat out City's last four Premier League fixtures, but he did feature during a recent Champions League match against Czech side Viktoria Plzen.
Watch Tony Cascarino's Monday Inquest on The Times Goals, which brings subscribers Premier League goals and highlights before anyone else on mobile, tablet and online. Visit www.TheTimes.co.uk/football to find out more.