The Black Cats have struggled in the Premier League this season and are 19th with only one point after failing to win any of their opening three matches.
"If we are a united group, we can achieve a lot more and, for that reason, we have to mix and share as much as possible," he told the Daily Star. "It is only natural that players who speak the same language will want to talk in that language.
"But as a group we must speak English and it's important the players who do not speak English learn as quickly as possible. I will make sure we get that togetherness and do not split up into little groups.
"We stay and talk – English with French, Irish with Swedish and so on. We are working very hard here to create the best environment."
Sunderland return to Premier League action with a home match against Arsenal next weekend.