The striker found the back of the net in the third minute of stoppage time to steal all three points for the home side at Stade Auguste Delaune to move them into the Europa League spots in the table.
© Getty Images
It was the visitors who had the first good chance on goal in the 10th minute. Didier Digard picked up the ball outside the box and played a lovely one-two with Dario Cvitanich, before hitting his effort straight down the throat of Kossi Agassa.
Two minutes later Nampalys Mendy raced to the edge of the box for Nice, but he wildly lashed his shot high and wide of the target.
Floyd Ayite came close to an opener for the hosts in the 19th minute, but the acute angle got the better of him and he flash a shot wide across the face of goal.
Despite Nice largely controlling the remainder of the first half, Reims has two good chances to snatch the lead before the break.
Chances were few and far between for the majority of the second half, but Prince Oniangue came close with a header that fell just wide of the far post from another superb Albaek cross.
Reims keeper Agassa then kept his side level with 15 minutes to play as he got down low to his right to keep out a stinging shot from range from Eric Bautheac.
However, Charbonnier tapped in a cross from five yards out in the dying seconds to consign Nice to their seventh consecutive defeat.
Reims move up to sixth in Ligue 1 with the victory, while Nice stay 17th.