Hilton scored a 29-yard touchdown over Browner in his side's win over the Seahawks and also had a career-high 140 receiving yards, as well as another score.
"Browner said I'm light as (expletive)," Hilton told USA Today Sports.
"He made me hit the light switch that nobody's seen. He made that decision. So they should blame him."
The Colts moved to 4-1 in their season with the victory over the Seahawks.