Michy Batshuayi had earlier ended his 11-game wait for a goal as Chelsea took a lead that they would ultimately squander, with Jamal Lewis popping up in the fourth minute of second-half stoppage time to rescue a 1-1 draw.
The Blues, held to a goalless draw in the reverse fixture 11 days ago - now one of five successive stalemates since the turn of the year - were then denied a penalty in a dramatic period of extra time when Willian was tripped in the box.
Despite their two sendings-off, Chelsea plugged away and came out on top in the shootout after converting all five of their spot kicks to survive a major scare against the Championship club.
Willian had another chance to open the scoring but could only drag wide after bringing down Cesar Azpilicueta's trademark diagonal pass, and the Spanish defender also came close when sliding in to meet a corner at the back post that narrowly evaded him.
Norwich, winless in 14 games against Chelsea heading into this latest encounter, were being pegged back and relying on Gunn to keep them in the contest.
The English keeper got a fingertip to Danny Drinkwater's 30-yard shot and helped it against the crossbar, as the home side's wait for a goal continued.
Second best up until that point, City did well to grow into the match and themselves hit the frame of the goal through Nelson Oliveira, who was gifted the ball on the edge of the box by a misplaced Batshuayi pass.
Norwich were looking composed on the ball before being dealt a sucker punch 10 minutes into the second half when Kenedy got in behind and squared for Batshuayi, giving the striker a simple finish into the roof of the net for his first goal in three months.
Aiming to prevent their five-match unbeaten run from coming to an end, City responded well and hit the post on the hour, with Josh Murphy's shot beating Ethan Ampadu on the line but coming back off the upright.
Substitute Morata almost made sure of Chelsea's path through to round four when glancing Drinkwater's cross wide soon after coming on, but it would prove to be a nervy finish for the home side.
After Morata and Willian failed to make contact with their chances from close range, Norwich snatched a leveller in the final seconds of added time through young defender Lewis, who glanced a header into the far corner for his first senior goal.
The first act of extra time for Graham Scott would prove to be contentious, as he cautioned Willian for diving - much like Pedro in normal time - but on this occasion the referee appeared to make the wrong call, as Willian was clipped by Timm Klose.
With the VAR call also going against the English champions, even more disappointment was to follow as Gunn produced another couple of saves in quick succession to thwart Willian and Morata, the latter volleying at the keeper from close range.
Chelsea had to see out the final stages of the contest with nine men after Pedro, booked for that earlier dive, was shown a second yellow late on for going through Wes Hoolahan, while Morata was also booked for diving and again for dissent in the space of a few seconds.
Klose headed the ball into the hands of Caballero from the final act to signal penalties, which would go the home side's way as Oliveira - first up for City - missed the only spot kick on a night of talking points aplenty.
CHELSEA (3-4-3): Caballero; Azpilicueta, David Luiz, Ampadu (Christensen 81'); Zappacosta, Bakayoko, Drinkwater (Hazard 100'), Kenedy (Kante 86'); Willian, Batshuayi (Morata 81'), Pedro
NORWICH CITY (3-4-3): Gunn; Hanley (Cantwell 86', Stiepermann 120'), Zimmermann, Klose; Pinto (Tettey 116',), Reed (Hoolahan 82'), Vrancic, Lewis; Maddison, Oliveira, Murphy