The 20-year-old recently announced himself by scoring the winner in a 2-1 victory over Manchester United at Old Trafford.
Berahino has also found the net for the England Under-21s against Scotland, Moldova and Lithuania, leading to suggestions that he is a dark horse to make Roy Hodgson's World Cup squad.
"Six weeks ago nobody spoke about Saido and now people want to put him in the World Cup squad," Clarke told reporters. "Come on, you have to keep a sense of realism.
"Saido's had a fantastic start to the season. He's scored some goals for the England Under-21s against opposition that is not the strongest. He's scored a Premier League goal for us and he's doing all right.
"Let's just give him time to develop and see where his career ends up. At the moment [I want] everybody to back off Saido and give him time to grow as a player. That's what I want more than anything. I want to give the boy time to grow and develop as a player."
Berahino has spent time on loan at Northampton Town, Brentford and Peterborough United since signing professional papers at The Hawthorns two years ago.