Nice had the best chance in the early stages when Mahamane Traore raced clear after 13 minutes but he could only put his shot wide of the post.
Three minutes later the home side were ahead, Diego taking advantage of some great play from Antoine Devaux to finish well.
Nice continued to attack but were unable to hit the target. Dario Cvitanich fired wide from a good position in the 26th minute.
Just before half-time, Valentin Eysseric found himself in a great position but an excellent block preserved Reims's lead.
Nice were finally level after 62 minutes, this time Cvitanich making no mistake after being set up by Eysseric.
It turned into an end-to-end encounter, but on 77 minutes Cheikh M'Bengue's shot was well-blocked.
However, with three minutes to play Reims got themselves ahead through a Gaetan Courtet strike.
It got better for the home side as Courtet scored another late on.