The visitors surged ahead at Stadium Municipal as early as the first minute when Eric Bautheac shot home from close range.
However, that lead lasted just seven minutes as Wissam Ben Yedder found the target for Toulouse from a difficult angle on the left.
The hosts then went ahead on 20 minutes through Etienne Capoue, Yedder releasing his teammate with a through-ball to slot home from the centre of the box.
Nice, went in the break level, thanks to Bautheac's second goal of the evening five minutes before the half-time interval.
The hosts went down to 10 men in the 83rd minute when goalkeeper Ali Ahamada was sent off for a bad challenge on Traore.
However, that did not stop Yedder from scoring his second goal of the evening and drawing level in the 90th minute.
There was yet more drama to come at Stadium Municipal, as in the second minute of injury time Nice's Alexy Bosetti was dismissed for a bad foul and two minutes later Kevin Anin shot home to give the visitors all three points.