Managerless Huddersfield started the tie the brighter of the two sides and went on to break the deadlock with five minutes on the clock. Former Leeds United midfielder Adam Clayton was the scorer, sending a free kick from 25 yards out over the wall and into the net.
Their advantage would only last for two minutes, though, as the hosts equalised almost immediately. The Huddersfield defence failed to clear a header from Ritchie de Laet and Manchester United loanee Michael Keane took full advantage to poke the ball in from close range.
Having drawn themselves level, Leicester began to dominate proceedings but they were unable to find the net before the break.
However, with 15 minutes remaining, the Terriers went in front for a second time. Sean Scannell picked the ball up just inside the Leicester half, before going beyond Paul Konchesky and Matt James to the lift the ball into the goal.
That promoted a series of attacks from the home team, but their search was ultimately unsuccessful.
Huddersfield will now take on Wigan Athletic at home for a place in the quarter-finals.