First, Taylor overruled his assistant, who had waved for a foul after Andy Carroll struck Simon Mignolet in the face, causing the Belgian to drop the ball and allow Guy Demel to poke home.
In the second half, Taylor awarded Liverpool their second penalty when Adrian challenged Jon Flanagan and looked have made contact with the ball.
Speaking after the game, Allardyce said: "There were fouls given by the referee when he was 60 yards away, the second penalty was not a penalty, Adrian plays the ball. Controversially we are talking about the referee and the effect he had on the game.
"He made big mistakes in the game. He thinks Adrian hasn't played the ball and he has. He has to be 100% certain. He was in a bad position but gave it.
"It looked like Andy Carroll fouled the goalkeeper for our goal but we got it. We don't want that, nobody wants that. My real concern is that we are talking about referees every week. We don't have enough quality referees. They try their best but something has to be done about it."
Defeat leaves West Ham in 11th, 10 points clear of the drop zone and three points shy of the top half.