The visitors did not need long to break the deadlock as Wes Burns sprinted into the box before slicing a powerful finish into the bottom corner.
Cheltenham continued to look the more threatening as Shaun Harrad tried to head it in, while Haynes had another chance but scuffed his finish inches wide.
Mark Byrne had the first real chance of the match for the hosts with just over half an hour gone when he tried to blast the ball in from a free kick, but the shot was tipped behind.
Newport did get back on level terms before the break though, as Jackson curled the ball in for Miles Storey to head into the bottom.
Burns tried to halt the Newport dominance with a charge down the other end, but his right-footed strike from the centre of the box could not beat the keeper.
Cheltenham looked the more likely in the closing stages, but were unable to force a late winner to move them away from the relegation zone.
The result leaves Cheltenham 22nd in the table, just two points clear of the relegation zone, while Newport stay ninth.