In the sixth minute, York's Wes Fletcher attempted a shot from outside the box, but his right-footed strike flew too high and too wide.
Drama ensued just a few minutes later when Neal Trotman was sent off for the hosts for a tackle on Ryan Bowman in the box. Fletcher's ensuing penalty then sent goalkeeper Jake Cole the wrong way for the game's opening goal.
Midway through the first half, the referee waved away a Plymouth shout for a penalty after John McCombe tangled with Reuben Reid, while on the half-hour mark, Conor Hourihane's long-range effort flew just wide.
The Minstermen doubled their lead on the stroke of half time when Bowman's cross fell to McCombe, who made no mistake with a close-range header.
Just before the hour mark, a good fingertip save by Cole from Will Hayhurst's shot kept the score down to 2-0.
This moderate scoreline wasn't to last, however, as two York goals right at the death, courtesy of McCombe's second strike and Josh Carson's injury-time effort, further compounded the home side's misery.