Conor Hourihane came close to opening the scoring just 12 minutes into the match, blasting a 35-yard effort towards the goal, but it was held by Newport's keeper.
The visitors were able to break the deadlock with just over 20 minutes gone, as Tyler Harvey headed in from a Nathan Thomas cross.
Plymouth's lead lasted just over five minutes as Newport found the back of the net against the run of play through a Chris Zebroski strike.
The visitors soon restored their lead though, as Hourihane drilled the ball in from just outside the box following a quick move from Reuben Reid.
Robbie Wilmott had a chance to get the hosts back on level terms just five minutes into the second half, but his effort from just outside the area was blocked.
The result leaves Plymouth ninth in the table with a game in hand, while Newport sit 14th.