Farmer was speaking after Manziel took third-string reps on Wednesday, with Hoyer and Tyler Thigpen above him in the depth chart.
Manziel himself has admitted that he faces a fight on his hands to become the team's starter.
"I want the best quarterback to start," Farmer told WKRK-FMW radio. "Who is the best quarterback? Brian Hoyer.
"It's a good question and I would tell you that Hoyer is better than Manziel, and probably by a substantial margin. I would say that because here's a guy that's a five- or six-year veteran who has been in the league, that knows the defenses, that reads it quicker, that understands it.
"Hoyer has an advanced calculus degree. Johnny is good at math, but I don't know if he's in the advanced calculus stages. Experience in the league means a lot. It means more than people think."
Hoyer threw five touchdown passes in his three starts for the Browns last season.