Brazilian striker Ilan pounced in stoppage time to make it two wins out of two for the newly-promoted Corsican outfit.
Hantz said that he is at a loss to explain his side's poor start to the match, but believes that the win will be a boost to the confidence.
"It was a scenario worthy of one of the great suspense films, but a magnificent one for our first home game since returning to Ligue 1," said Hantz.
"The result is very important for us in terms of confidence. However, I cannot explain our very laborious and difficult start to the match. Was it down to the heat or was it an athletic problem? I can't say at the moment.
"It was a similar evening to last weekend at Sochaux where we went behind before coming back to win the game. It doesn't happen very often in modern football and I am satisfied."
Bastia travel to Rennes next Saturday.