However, Robert Snodgrass quickly equalised and after they held on for the remainder of the match, Hull prevailed in the shootout as Newcastle missed three of their four kicks.
Read below to see how the action unfolded at the KCOM Stadium.
Last month, Hull were able to progress without scoring any late goals as efforts either side of half time from Harry Maguire and Michael Dawson proved too much for Bristol City.
After 2-0 wins over both Cheltenham Town and Wolverhampton Wanderers, the Magpies thrashed Preston North End 6-0 in the last 16. Preston weren't helped by the first-half dismissal of Alan Browne but it was a ruthless performance from the North-East giants with Mitrovic and Diame both netting doubles.
17 - The last four meetings between Hull and Newcastle have produced 17 goals in total (4.25 per game). Open.— OptaJoe (@OptaJoe) November 29, 2016
Moments after that, Diame - playing against his former club - fired marginally wide from 20 yards out.
After Lazaar had struck the crossbar, Newcastle opened the scoring from a corner with Gayle's flick on finding Diame, who netted from close range against his former club. However, just 77 seconds later, Hull found an equaliser through Snodgrass, who fired in the rebound after Henriksen's shot had been parried by Sels.
What an unbelievable sequence of events, especially given that this match has been rubbish.