The former Liverpool boss has struggled to strike up a relationship with the majority of Blues fans, who continue to boo the Spaniard following poor results, since replacing Roberto Di Matteo last November.
"That falls on us because if you are winning games, that gets taken away," Terry told The Independent. "If you are losing games, the pressure is on the manager. He has to take responsibility because he is the one who picks the team.
"So if we are not [winning], naturally with all managers, it falls on his head unfortunately. [The fans] pay their money so they are entitled to echo their thoughts. All we can do is concentrate on our jobs and that is winning games. If we can do that, it completely takes that away."
The Blues will go up against Manchester United in the quarter-finals of the FA Cup if they defeat Middlesbrough in the fifth round, while they currently sit in third place in the Premier League table.