The Tigers were reduced to 10 men in normal time when Dieumerci Mbokani was sent off but after Robert Snodgrass cancelled out Mohamed Diame's opener, they dominated from the spot as Newcastle missed three of their four attempts from 12 yards, with Eldin Jakupovic saving from both Jonjo Shelvey and Yoan Gouffran while Dwight Gayle struck the crossbar.
Both sides began the match at the KCOM Stadium intent on keeping possession over attempting to create anything on goal, and it took until 13 minutes for the first effort on target to come from Newcastle.
It would have been a fine goal, too, with Shelvey's exquisite through-ball finding the run of Gayle but the forward put his half-volley too close to goalkeeper Jakupovic.
Moments later, Diame - playing against former club Hull for the first time since his exit in the summer - fired wide from 20 yards before the Tigers finally registered a shot on goal in the 23rd minute when Snodgrass lashed wide.
Newcastle were soon back on top as Diame headed over the crossbar from eight yards out, while Matt Ritchie came within inches of curling the ball into the top corner from 20 yards.
Isaac Hayden - another former Hull player - blasted marginally wide of the far post from 20 yards but it proved to be Newcastle's last chance in a half which they dominated without breaking the deadlock.
After the restart, Newcastle remained on the front foot but they were unable to create any more opportunities and that allowed Hull to finally get a foothold in the game.
However, they also struggled to threaten an opening goal and aside from a Snodgrass free kick which went a yard over the crossbar, they had no joy in trying to break the deadlock.
Extra time was effectively an inevitability but with a minute of normal time remaining, Hull were reduced to 10 men when Mbokani was sent off for his involvement in an incident with Jamaal Lascelles, with the striker kicking out and pushing his head into the defender.
The dismissal handed Newcastle the advantage and they made the most of it during the early stages of extra time, with Achraf Lazaar almost opening the scoring with a left-footed shot that hit the underside of the crossbar.
Hull were fortunate to survive but it was for less than a minute as Newcastle finally went ahead through Diame, who tapped home from close range after getting on the end of Gayle's header.
The Championship side should have had enough to see out the remainder of the match but just 77 seconds later, Hull equalised when Snodgrass was on hand to fire home the rebound after Matz Sels had parried from Markus Henriksen's well-struck shot.
Despite conceding against the run of play, Newcastle recovered to create more chances against Hull's 10 men, but Gayle was guilty of failing to make the most of a header from eight yards out and a first-time strike from inside the penalty area.
Hull also had their moments during the last 10 minutes and Henriksen should have done better with a curling effort from around 14 yards.
However, the Tigers were still relieved to make it through extra time on level terms and they capitalised on reaching a penalty shootout by taking advantage of misses from Newcastle trio Shelvey, Gayle and Yoan Gouffran.