The visitors failed to make the most of their first-half dominance against their Premier League rivals and were once again made to pay for their profligacy when Shinji Okazaki came off the bench to open the scoring shortly after the hour mark.
Islam Slimani then added a stunning second to seal the win for Leicester, condemning Liverpool to a fourth consecutive match without a win for the first time since February.
Both managers took the opportunity to make sweeping changes to their teams, including Leicester debuts for Aleksandar Dragovic and Vicente Iborra and a first Liverpool start for deadline-day signing Alex Oxlade-Chamberlain.
Leicester failed to deal with the resulting corner too and the ball eventually fell to Dominic Solanke, who flashed a snapshot right across the face of goal on the turn.
It proved to be a sign of things to come in the first half as Liverpool continued to knock on the Leicester door, and Ben Hamer was called into action for the first time shortly afterwards when Philippe Coutinho danced his way into a shooting position before drawing a save from the keeper.
Hamer made a more routine stop to deny the Brazilian six minutes later before Danny Ward was finally forced into his first piece of action on the night, although that too was a simple save to keep out a tame Demarai Gray strike at the end of a counter-attack.
Robertson was one of Liverpool's main attacking outlets as he continually found space down the left, and one of his teasing deliveries almost set up a chance for Solanke, who could not hook his effort on target.
The full-back was close to providing the opener again shortly before the half-hour mark when he fired a powerful ball across the face of goal, but a slight touch off Leicester skipper Wes Morgan proved crucial as he denied Coutinho a simple finish into the empty net.
Oxlade-Chamberlain and Coutinho then both failed to truly trouble Hamer with efforts from outside the area as Liverpool - who had reached at least the semi-finals of this competition in each of their last three attempts - continued to look for a deserved breakthrough.
Leicester held on until half time, though, with Solanke squandering the visitors' final chance of the half in stoppage time when he latched on to Coutinho's exquisite pass only to lift his finish over the crossbar.
Despite Coutinho's influential first-half display, Klopp opted to withdraw the Brazilian at half time as Ben Woodburn came on in his place, and Liverpool's control of the game diminished as a result.
Okazaki's introduction for the injured Leonardo Ulloa saw Leicester grow as a threat too, and after Gray had dragged one wide of the target the Japanese striker broke the deadlock with a scuffed finish which bobbled into the bottom corner.
Liverpool immediately went searching for a route back into the game and came close through Oxlade-Chamberlain and Woodburn before Klopp turned to Danny Ings, who came off the bench for his first appearance in 11 months following a serious knee injury.
The striker had the chance to make it a dream return just four minutes after his introduction when his dart to the near post saw him threaten to get on the end of Woodburn's cross, but he could not get anything on his attempted unorthodox finish.
It proved to be a pivotal moment in the match as Leicester doubled their advantage moments later, and while the first goal was scrappy, the second was sublime as Slimani blasted the Foxes to within touching distance of the fourth round.
Okazaki was again involved as he nudged the ball into the path of the club-record signing, who powered past the booked Marko Grujic before unleashing an unstoppable effort into the top corner.
The hosts almost added a third - and a second in as many minutes - when Gray was only denied by a fingertip stop from Ward, before the Liverpool goalkeeper made a more routine save to deny Okazaki.
The damage had been done by that stage, though, as Leicester saw the game out for their first win in four matches, keeping their first EFL Cup clean sheet since August 2012 in the process.
Liverpool, meanwhile, have now failed to score in back-to-back away games for the first time since December 2015, with the eight-time champions crashing out in the third round for the first time since 2013.
Klopp's side will not have to wait long for a chance to take revenge, though, as they return to the King Power Stadium for a Premier League match on Saturday.