The Champions League hopefuls looked to be on course for a share of the spoils in Florence, but Paulo Sousa's side struck in the dying stages to clinch a 2-1 win and climb above Roma into third in the Serie A table.
The home side made a positive start to the match and threatened to break the deadlock early on when Federico Bernardeschi danced his way into a shooting position, only to fire straight at Samir Handanovic.
The Inter goalkeeper was called into action again shortly afterwards as Fiorentina continued to push for an early breakthrough, but once again it was a fairly comfortable stop to this time deny Josip Ilicic.
Ilicic had another sight of goal thwarted by a last-ditch block with a just under 15 minutes on the clock, but it would be the visitors who broke the deadlock with their first clear opening of the contest.
Fiorentina almost came up with a response right on the stroke of half time when Matias Vecino climbed highest in the area to steer a header towards the corner, but Handanovic was equal to the effort and turned the ball behind.
There was nothing the Inter shot-stopper could do to keep out Fiorentina's equaliser on the hour mark, however, as Borja Valero levelled things up with a well-placed header just inside the left post.
Inter were forced to see out the match with just 10 men when Alex Telles was shown a second yellow card in the 83rd minute, and their numerical disadvantage eventually told when the hosts nicked a winner in stoppage time.
It was Babacar who got the goal just a few minutes after coming off the bench, sweeping home a rebound after Inter had failed to clear their lines during a goalmouth scramble.
Fiorentina also ended the match with only 10 men on the field due to a straight red card for Mauro Zarate in the fifth minute of stoppage time, but Inter did not have long enough to take advantage as their poor run of form stretched to just one win in seven.
Indeed, things got even worse for the visitors before the final whistle as they were shown another red card with virtually the last action of the match as Geoffrey Kondogbia was handed his marching orders.