Hoyer replaced starter Brandon Weeden in the lineup after the 2012 first-round Draft pick sustained a thumb injury and the former Patriot threw touchdown passes in his side's win over the Minnesota Vikings.
"He has everything it takes physically and mentally to play the position," said Brady told Cleveland.com. "He loves football and works hard at getting better.
"We talked football whenever we were around one another, but he was solely responsible for his work ethic and improvements over the years."
The Browns face the Cincinnati Bengals on Sunday.