Both sides ended the match with 10 men in a dramatic final few minutes that also saw Antonio Candreva miss a penalty, only to score the winner from the rebound.
The match was just five minutes old when the opening goal arrived, and it came in style as Candreva gave the visitors the lead with a fine strike from the edge of the box following a well-worked corner routine.
That proved to be the highlight of a largely uneventful first half, although Candreva did have a glorious chance to double his side's advantage right at the end of the opening period, only to blaze over from close range.
Inter Milan finally produced their first meaningful attacking moment of the game shortly after the hour mark, and it provided an equalising goal too as Mauro Icardi latched on to Ivan Perisic's through-ball before keeping his composure to slot past Etrit Berisha.
The visitors were handed a glorious chance to regain their advantage with just three minutes left when Felipe Melo conceded a penalty, and Candreva doubled his personal tally at the second attempt, tucking home the rebound after Samir Handanovic had saved his initial spot kick.
Melo's evening went from bad to worse late on as he was shown a straight red card in the 90th minute, all but ending any hopes of Inter getting back into the match.
Five minutes of stoppage time, plus a second yellow card for Lazio's Sergej Milinkovic-Savic gave the hosts a glimmer of hope, but the capital outfit held on for a victory that sees them climb up four places into the top half of the table.