The loss at the Etihad Stadium means that the Premier League champions have now conceded in all seven of their matches this season.
"It concerns us because a team like Man City cannot expect to concede four goals at home," Toure told CityTV.
"Since the season started we have not had a clean sheet and that is very important when you want to be a top club, you have to defend better as a team.
"As a defender and as a group we have to work on it because mistakes were done by a few players and when you win it is everybody so when you lose it is everybody."
City have the chance to earn their first clean sheet of the campaign on Saturday when they travel to Fulham.