The result has seen the Panthers finish in first place place in Group D, two points above Congo DR.
Daniel Cousin hit the post for Gabon 10 minutes before half time, and they were made to pay on 64 minutes thanks to Moulaye Ahmed's calm finish as Mauritania regained the lead.
Gabon nearly got themselves level with 15 minutes left on the clock, but Romuald Ntsitsigui rattled the crossbar with his effort.
Needing a goal to reach the last eight, Gabon found the net with four minutes left when Aaron Ondele got on the scoresheet with a powerful header.
They then clinched the victory deep into stoppage time as Bonaventure Sokambi scored two goals in quick succession, the latter of which was from the penalty spot.