Speaking afterwards, Pardew said: "We left ourselves open at times when we were chasing. It was a really good game. We were pressing.
"We knew if we could get Papiss on and if he could get a chance he'd score. I'm pleased for the players but disappointed for the club because I thought we deserved more."
Pardew also commented on the state of his centre-backs, an area he has been struggling with of late.
"[Fabricio] Coloccini is a bit sore. Whether he did enough to play on Saturday I don't know. I'm not sure I should have played [Mike] Williamson. But we're short at centre-half," he added.
United went on to draw Chelsea in the fourth round.