© Getty Images
Mustapha Diallo spurned a great chance for Guingamp after 15 minutes when he sent his header wide of goal having been found by a ball from Rachid Alioui.
Romaric was at fault for Bastia as he wasted a good opportunity to give them the lead by somehow heading over from close range with only the keeper to beat.
Wahbi Khazri let fly with a powerful effort in the 36th minute, but it was kept out by a fine save from the visiting goalkeeper.
Guingamp went in front after half time when Alioui turned in Younousse Sankhare's cross, but Bastia levelled through Adama Ba, who smashed his effort into the top corner after being picked out by Djibril Cisse.
The visitors then went back in front as Alioui notched his second goal of the night with a neat conversion. However, Squillaci popped up in the final stages of the game to head home and put the hosts back on level terms.
The drama wasn't over yet, though, as Boudebouz stepped up with two minutes remaining to convert a penalty and hand Bastia all three points.