The hosts began brightly and weighed in with the first chance of the game on three minutes, Etienne Didot volleying over the bar from the edge of the box.
PSG then countered with a dangerous fifth-minute effort which saw Angel Di Maria cut inside and curve a shot just inches over the Guingamp crossbar.
On 14 minutes, Gianluigi Buffon pulled off a stunning save from a Nolan Roux effort, pushing the ball onto the bar despite already diving the other way, and at the other end, Di Maria came close with a cheeky backheel.
Guingamp gave as good as they got in the early stages of the encounter and went on to take the lead in the 20th minute as a lack of communication between Colin Dagba and Thiago Silva allowed Marcus Coco to steal the ball and set up Nolan Roux for a simple finish.
The hosts pressed on against the beleaguered visitors and were unlucky to see a 29th-minute goal ruled out when Nicholas Benezet headed home from a few yards out as the referee deemed Dagba to have been pushed in the build-up.
Indeed, PSG were lucky to go into the break just a goal down, but they successfully regrouped at the interval and drew level on 53 minutes through a Neymar penalty.
Guingamp were on the back foot at this stage but continued to hold on, until Kylian Mbappe broke their resistance eight minutes from time, latching on to Di Maria's pass and sliding home.
The France starlet, who came on as a substitute, made sure of all three points in the 90th minute with a cool chip over Karl-Johan Johnsson from just a few yards out.
GUINGAMP (4-2-3-1): Johnsson; Ikoko, Kerbrat, Eboa, Rebocho; Didot (Sorbon 81'), Blas; Coco, Benezet (Rodelin 66'), Thuram (Salibur 75'); Roux
PSG (4-3-3): Buffon; Dagba, Marquinhos, Silva, Meunier; Bernede (Lo Celso 80'), Nkunku (Diaby 74'), Rabiot; Di Maria, Weah (Mbappe 46'), Neymar