Franck Tabanou and Ismael Diomande had given the hosts a comfortable lead either side of the break, before goals from Gianni Bruno and Francois Modesto earned the visitors an unlikely point at Stade Geoffroy-Guichard.
The home side found an opener with 28 minutes of the clock when Tabanou ran onto a high ball over the top before smashing a volley in the top corner of the net from the edge of the six-yard box.
Les Verts doubled their lead in style just short of the hour mark when Diomande powered a loose ball into the bottom corner from the edge of the box.
The visitors made sure of a nervy finish to the match when Bruno connected with a fine header at the near post to pull a goal back for Bastia.
Bastia completed a remarkable comeback in the fifth minute of stoppage time when Modesto guided home a Julien Sable cross into the box despite appearing to be in an offside position.
Saint-Etienne remain sixth in Ligue 1, while Bastia climb up to 11th.