The second half saw few chances arrive for both teams, Bradley Dack having the best opportunity after Argyle keeper Luke McCormick dropped a corner, but the midfielder was unable to direct the ball into the net.
Dack then had an effort ruled out for offside before Craig Tanner gave Plymouth the lead with an unstoppable volley after a corner had been half-cleared by the Gills defence.
Dack continued to pull the strings for Gillingham and he was rewarded when a deflected pass ended up in front of him to shoot past McCormick and equalise for his side five minutes from time.
Jake Hessenthaler then gave Gillingham a dramatic winner after nodding in from close range.
Gillingham's next match will be a League One fixture away to Port Vale on August 15.