Bastia started the stronger side, with Wahbi Khazri and Julian Palmieri both firing wide inside the opening 15 minutes.
In the 17th minute Bastia were ahead after Rincon put through his own net, but Marcos equalised after 26 from a corner.
Troyes pushed to go ahead and Marcos went close with another header on 30 minutes but it went wide.
Bastia were back ahead before half-time when Anthony Modeste struck from a Khazri assist on 37 minutes.
Marcos grabbed his second and another equaliser just after half-time from the penalty spot after Jeremy Choplin was booked for a foul.
However, Modeste missed from the spot and the score remained 2-2.
Bastia finally managed to make their advantage count with two minutes on the clock as Toifilou Maoulida found the net from a header.