The Potters, who were relegated from the Premier League last term, are yet to win a game since the former Birmingham City and Derby County boss took the reins in May.
On Wednesday, Stoke were humbled in front of their fans as they lost 3-0 to Wigan Athletic at the bet365 Stadium.
"I knew it would not be an easy job. We have to try to change the culture very quickly," Rowett told BBC Sport.
"I would say we have got to stick together as a club but, in all fairness, if you are a fan, you would be wavering on that a little bit at the moment, because you have not had any value for money for quite some time.
"What we have got to do is earn that trust back. We have got to start performing, stop talking about it and do it on the pitch."
Stoke are at home to Hull City on Saturday, before Tuesday's visit of top-flight Huddersfield Town in the second round of the EFL Cup.