Protests from fans have been common in recent weeks due to the team's poor form, but Allardyce believes that it is only a small number who want to see him replaced at Upton Park.
When asked about the majority of supporters not being behind him, the Hammers boss responded: "Majority? Minority. Minority.
"Let me put it right - a minority. Because the majority who are satisfied do not ring up, write in or come on the radio.
"They go home and they wait and hopefully next week they find it better."
Allardyce claimed on Thursday that he has no plans to walk away from West Ham.