Rangers took just 68 seconds to score as Ryan Manning headed home from a pinpoint Ilias Chair cross.
Argyle, who hit the post through George Cooper, replied when on-loan Blackpool striker Ryan Hardie set up midfielder Joe Edwards to sweep home past Joe Lumley after 32 minutes.
The newly promoted League One side went 2-1 up in the 55th minute when Danny Mayor beat his marker and smashed home from inside the box.
The visitors' response was immediate with a thumping Osman Kakay drive beating Michael Cooper.
Cooper made superb saves to deny Manning and Bright Osayi-Samuel before central defender Scott Wootton came to the keeper's assistance to clear Aramide Oteh's goal-bound shot off the line.
Wootton again cleared off the line when substitute Paul Smyth's effort beat Cooper and came off the post with 15 minutes remaining.
Three minutes later former Colchester striker Nouble met Conor Grant's pacy cross with a first time aerial volley to secure Argyle the win.