Hoyer completed just 16 of his 41 passes and he received the lowest passer rating of his career at 46.3, 20 points lower than his previous worst.
"I'm just angry," Hoyer said, according to Akron Beacon Journal.
"Even my wife says she doesn't even want to be around me. She doesn't want to be around me when we win, too, because I'm so critical."
Despite his struggles against Jacksonville, coach Mike Pettine has insisted that Hoyer will remain as his starting quarterback against the Oakland Raiders on Sunday.