The Dutch conceded two questionable penalties against both Spain and Australia and, despite eventually emerging victorious from those encounters, Van Gaal has called for officials to be regularly assessed to raise standards.
He told reporters: "The first two matches, we were faced with two unfair penalties. That could have led to a disqualification.
"I think the referees should also be evaluated and assessed -- the media assess and evaluate players, but you should also go for the referees.
"If they have made a mistake they should be put at a lower level. That's what I do with my players. If they don't perform, if they don't play their match very well, the next match I will choose a different player, possibly. If only FIFA would do the same thing."
Holland face Mexico in the last 16, with the tie taking place in Fortaleza on Sunday.