Brendan Rodgers appeared to have masterminded a smash-and-grab draw on his first return to Anfield since being sacked in October 2015 when James Maddison scored with Leicester's only shot on target to cancel out Sadio Mane's opener.
However, Marc Albrighton conceded a stoppage-time penalty for a foul on Mane, allowing James Milner to tuck the spot kick home and make it 17 successive Premier League victories, including eight from eight so far this term.
Even at this early stage of the campaign, the jubilant scenes at the final whistle hinted at the magnitude of a victory which takes them temporarily eight points clear at the top of the table as they inflicted only a second defeat of the season on Leicester.
The Foxes were one of only two visiting teams to take points off Liverpool at Anfield last season - alongside champions Manchester City - and came into the match sitting third in the table, so it was no surprise to see them make life difficult for their hosts.
It was Liverpool who looked most like scoring for the vast majority of the match, though, and the first half-chance fell to Mohamed Salah after four minutes when he span away from the close attention of Caglar Soyuncu but then put his finish too close to Kasper Schmeichel.
Trent Alexander-Arnold proved to be Liverpool's main attacking threat for much of the half and he created the first big chance of the contest when he skipped past England teammate Ben Chilwell and swung a cross towards Milner, whose volley bounced into the ground and over the crossbar.
Mane was the next to threaten when he drew a routine stop from Schmeichel on the half-hour mark, and another pinpoint delivery from Alexander-Arnold shortly afterwards created an opening for a stretching Roberto Firmino, who could not turn his shot on target.
Leicester did begin to grow into the game during the final 15 minutes of the half, but they struggled to ask enough questions of Adrian and their best chance of the opening 45 minutes - a Soyuncu header which was well saved by the Liverpool keeper - would not have counted anyway due to the offside flag being raised.
Just when the Foxes were enjoying their best spell of the match, Liverpool went up the other end and broke the deadlock. Milner released Mane through on goal with a brilliant left-footed pass forward, and the Senegalese winger made no mistake with his finish to bring up 50 Premier League goals in his 100th league game for the club.
Number 51 almost followed just three minutes later when Firmino cut the ball back to Mane, who this time could only steer his effort on target as Schmeichel made the save.
Milner then blazed one over the bar moments afterwards as Liverpool finished the first half strongly, and they quickly picked up where they left off at the start of the second half.
Alexander-Arnold was again involved as he poked the ball in to Salah, who was only denied by a fine reaction stop from Schmeichel, although replays suggested that any goal would not have counted anyway with Salah shooting from an offside position.
Jurgen Klopp's side came close again 10 minutes later when a string of one-twos between Salah and Mane almost released the latter into the box, with a last-ditch challenge from Chilwell bouncing against Mane's leg and flashing just past the post.
Alexander-Arnold then fizzed a pass into the feet of Firmino inside the area and the Brazilian instantly controlled it before dragging his shot wide of the far post.
The first warning shot that Liverpool's profligacy might come back to haunt them came midway through the second half when Albrighton's pass released Vardy through on goal, only for the striker's first touch to show too much of the ball to Adrian, who came out to smother it.
Liverpool were straight back up the other end just a minute later, though, as Salah poked the ball through for Andrew Robertson, who fired his finish straight down the throat of Schmeichel.
Georginio Wijnaldum was the next to create a sight of goal only to waste it with a tame shot, and soon after Leicester offered another glimpse of their increasing threat - although Dennis Praet's 73rd-minute piledriver which fizzed just past the post was their first shot of any note.
Nevertheless, Liverpool's control over the game was undoubtedly slipping and Leicester gained more and more encouragement heading into the final 10 minutes, particularly after Salah failed to make the most of two promising attacking platforms in quick succession.
Sure enough, the Reds were punished 10 minutes from time when Ayoze Perez slid a ball through for the returning Maddison, who put enough on his low strike to beat Adrian despite the Liverpool keeper getting a hand to the ball.
Conceding to Leicester's first and only shot on target of the match would have been a gutting blow for Liverpool and one which looked set to see them drop their first Premier League points in more than seven months, and Leicester may have even sniffed a win with the wind now in their sails.
The leaders had to wait until stoppage time for their moment as Mane won the ball inside the Leicester area and was then bundled into by Albrighton, causing referee Chris Kavanagh to point to the spot with the decision being upheld by VAR.
Milner stepped up and coolly sent Schmeichel the wrong way to send Anfield into rapturous celebrations as Liverpool provided further evidence of their title credentials.
Klopp's side could now equal Manchester City's record of 18 successive Premier League victories when they face bitter rivals Manchester United at Old Trafford after the international break, and will go into the second break of the season with at least a five-point lead at the top.
LIVERPOOL (4-3-3): Adrian; Alexander-Arnold, Lovren, Van Dijk, Robertson; Wijnaldum (Henderson 78'), Fabinho, Milner; Mane, Salah (Lallana 92'), Firmino (Origi 78')
LEICESTER (4-1-4-1): Schmeichel; Pereira, Evans, Soyuncu, Chilwell; Ndidi; Barnes (Albrighton 46'), Praet (Perez 73'), Tielemans, Maddison (Choudhury 86'); Vardybody check tags ::