The two sides meet in the World Twenty20 on Friday with the build-up having been dominated by comments made by James Faulkner, who revealed that he does "not particularly like" the defending champions.
However, Sammy has claimed that the Australian all-rounder may be on his own in his view of the West Indies.
"The Australians normally have a lot to say," the Windies skipper told reporters. "We are here to play cricket.
"I think probably James is the only cricketer that does not love West Indians. I could safely say West Indies are the second favourite team for the fans. It does not bother us.
"Talk is talk. We have got to talk the talk out there on the cricket field. He can say all he wants. We are not bothered by it."