Cisse fired a late winner to secure all three points for the Magpies, but Pulis claims that the striker should not have been on the pitch.
Pulis insists that Cisse should have been given a red card for raising his hands and catching Marc Wilson and Ryan Shawcross.
"The second, I am not so sure Papiss Cisse should have been on the pitch," Pulis told reporters.
"I know he scores but he raised his hands and caught Marc Wilson and Ryan Shawcross.
"But that is the way it is going for us and we are really disappointed and sums up our season away from home, where we have played really well at times and should have taken points and this is another game we have taken none from."
Stoke are currently in 11th place in the Premier League, level on 33 points with Newcastle.