Despite the final score, other results in Ligue 1 have ensured that both sides will avoid relegation from the division this season.
It took seven minutes for the game's first chance to unfold, but Reims's Floyd Ayite failed to hit the target.
Nicolas Fauvergue had a chance for the hosts shortly after, but his finish fell wide of goal.
Grzegorz Krychowiak nearly put Reims ahead just before the break, but his left-footed shot from outside the box went high and wide to the left.
Valenciennes went in front with 32 minutes left to play courtesy of a Pujol strike.
The visitors came close to doubling their lead 19 minutes from time but Agassa pulled off a stunning save from Remi Gomis's effort that was heading for the top corner of the net.