The Premier League champions have struggled of late and have failed to win their last three games, and Carragher believes that an ageing squad needs to be freshened up.
He told the Daily Mail: "[City] look like a team which must be changed and refreshed if it is going to carry on moving forward. Whether they successfully defend the title or fail to contain Chelsea, Manuel Pellegrini's squad needs to be broken up.
"With an average age of 28.5, City's first-choice team — Hart; Zabaleta, Kompany, Demichelis, Kolarov; Nasri, Toure, Fernandinho, Silva; Dzeko, Aguero — is the oldest in the Premier League. It is also older than the first-choices of Real Madrid (26.2) and Bayern Munich (27).
"Those two clubs are the ones City want to eventually rival but the thing that should provide most concern for Pellegrini is the fact that for all the money that has been invested, there is no exciting youngster ready to explode on to the scene like Adnan Januzaj or Raheem Sterling."
City host local rivals Manchester United on Sunday.