© PA Photos
However, it was the home side that had the majority of the possession and chances in the opening half, with Moussa Sissoko and Yohan Cabaye both going for goal from distance, before Asmir Begovic had to make a save to keep Davide Santon out.
The Magpies continued to dominate after the break, Cabaye, Santon and Jonas Gutierrez all with chances inside the opening 10 minutes.
Walters, who is the club's top scorer so far this season, took the spot kick himself, blasting the ball past Elliot to give the visitors the lead in the 67th minute.
The advantage was short-lived, as the hosts were awarded a free kick 18 yards out following a foul by Glenn Whelan, allowing Cabaye to curl the ball in off the crossbar directly from the set piece.
© PA Photos
The Frenchman had a chance to repeat the goal and give Newcastle the lead soon after, but his low free kick bobbled wide.
The hosts had a string of efforts in the final minutes of the game, but stoppage time was needed for Cisse to fire the ball in from close range and seal the victory for Alan Pardew's side.
The win lifts Newcastle to 13th in the table, while Stoke remain 11th.