The Red Devils have so far failed to meet expectations after struggling for consistency following a summer spending spree that totalled in the region of £150m.
United sit sixth in the Premier League table after dropping 12 points already, but Van Gaal claims to have an "iron shield" to protect him from the critics.
"Maybe you know that I am always a controversial person and a lot has been written about that, but I [have] an iron shield," he told Sky Sports News.
"It's been like that since I came to England. Everyone can write or speak about me and I cannot change that.
"It took a long time for me to understand but I cannot change that. I have learned not to react and to believe in myself, my players and staff. That's important."
Van Gaal and co. host league leaders Chelsea at Old Trafford on Sunday.