The 33-year-old came under fire after making two mistakes in the defending champions' opening 5-1 defeat against the Netherlands, while also struggling against Chile as his side bowed out at the group stage.
"In Brazil, it is said that when you win, everybody wins and loses together," Carlos, a former Real Madrid teammate of Casillas, is quoted as saying by Goal.com. "Iker, as has been spoken about, was not at his best, but the others didn't help him either.
"There are two people on the field who can do no wrong. The striker and the goalkeeper, when they make mistakes, it's very noticeable and gives way to criticism.
"Iker has to take responsibility. As a friend, I'm sorry for him and for the team, but you have to always improve as we do not live in footballing past, but in the present."
Carlos was a part of Brazil's 2002 World Cup winning squad.