The hosts were criticised for their display last Saturday on their way to success over Chile on penalties, and Scolari reacted angrily to questions about his style in the pre-match press conference.
"If you don't like my leadership, you can go to hell," the 65-year-old told reporters in Fortaleza.
Scolari was also eager to dismiss suggestions that he claimed he would like to swap a member of his squad for someone left out of the group of 23.
"That's not what I said," he added. "What I said was that at this time of the competition, I could add a player with different characteristics for the matches we have forthcoming. If you could ask all the coaches, for one reason or another, they would like to add someone."
Brazil will meet either France or Germany if they can edge out Colombia on Friday.