It was an even start to the game as both sides felt each other out early on. The only opening of the first 10 minutes fell Newcastle's way, with Papiss Cisse flicking a header just over the top.
Norwich then took control and began to dominate the contest. Dieumerci Mbokani had the ball in the net in the 17th minute, but the striker had strayed offside and the goal was chalked off.
Karl Darlow, playing in goal for Newcastle after Rob Elliot's season was ended over the international break, then made two fine saves to keep the scores level. First, he beat out Robbie Brady's effort before somehow getting his foot in the way of Mbokani's close-range strike.
It looked as if Newcastle had ridden the storm and would reach half time level, but Norwich then took the lead in added time when Brady curled over a free kick that Timm Klose did well to rise and meet, guiding the ball into the corner.
Cisse was chosen ahead of Aleksandar Mitrovic to start in attack for Newcastle and the former spurned a great chance to equalise shortly after the resumption when he smashed wide after John Ruddy parried out Andros Townsend's shot.
Mitrovic came on shortly after the hour and he made his mark within 10 minutes when his header looped up off Ryan Bennett and into the net.
The Magpies were level for just two minutes as Mbokani restored Norwich's lead by cutting inside from the left and rifling past Darlow high into the net.
Newcastle appeared to grab a point five minutes from time when Mitrovic tucked home from the penalty spot after Andre Wisdom was adjudged to have handled in the box, but that was not the last of the drama.
The visitors were buoyed by levelling and pushed for a winner, with Ruddy springing to his left to keep out a Cisse header before Norwich snatched the points deep in added time.
Olsson arrived in the box to meet Howson's lay-off and arrow a low shot into the bottom corner to send Carrow Road wild.
© Getty Images
It means that Norwich are four points clear of the drop zone and Newcastle are now six from safety.