The Canaries equalised twice during a thrilling first half at Carrow Road, with Todd Cantwell cancelling out Abraham's first before Pukki continued his fine start to the campaign by responding to Mason Mount's goal.
However, Abraham's second goal with 22 minutes remaining proved to be too much for Norwich to recover from as Chelsea held on for a 3-2 victory which gives them their first win of the campaign at the fourth attempt.
Norwich will take plenty of positives from their performance despite the result as they fell to their second loss of the campaign, but ultimately it was a deserved win for the visitors on a sunny afternoon in Norfolk.
Chelsea were dealt a blow even before the match kicked off when Pedro was injured in the warm-up, leading to Ross Barkley coming in and making it the Blues' youngest starting XI in a league match for more than 25 years.
The setback did not seem to affect the visitors in an attacking sense, though, and they took the lead inside four minutes when Cesar Azpilicueta - criticised for his defensive work in the opening two matches of the season - got down the right and played a cross into the box for Abraham, who lashed home his first Chelsea goal on the half-volley.
It was another quick start for the Blues, but the same defensive issues which have cost them so dearly soon arose again as Norwich equalised just 156 seconds later.
Cantwell assisted two of Pukki's three goals against Newcastle United last weekend, but the roles were reversed this time around as the Finnish striker's low pass into the box was well converted from close range by the young Englishman.
The pace of the match showed no signs of relenting after the opening two shots had led to two goals, and the theme of young English players impressing also continued as Chelsea restored their advantage in the 17th minute.
This time it was Mount who made it two goals in his last two games with a well-taken strike, cutting inside before firing his effort past Tim Krul into the far corner after Norwich had given the ball away when trying to play out from the back.
The Canaries' refusal to abandon their attacking instincts almost came back to hurt them again shortly afterwards when Andreas Christensen stole in behind and met a cross with a firm header, but this time Krul pulled off a fine reaction stop to keep the deficit at one.
Ben Godfrey then threw himself in the way of a Jorginho effort inside the area before Norwich went up the other end and levelled things up for a second time, with Pukki continuing his free-scoring start to the season.
Emiliano Buendia slipped a pass through to the in-form striker, who turned it past Kepa Arrizabalaga from a tight angle for his fifth goal of the campaign - equalling Pavel Pogrebnyak's record for most goals from a player's first three appearances in the Premier League.
Buendia had a long-range effort of his own pushed away before Barkley blazed a presentable opening over the crossbar just before half time as the two sides went into the break all square.
Chelsea were once again quick out of the blocks after the restart with Mount dragging an effort wide and Christian Pulisic then slamming a shot into the side-netting, and this time the Blues managed to maintain a degree of control over the game.
Jorginho, Azpilicueta and Mateo Kovacic all had sighters in the opening 10 minutes of the second half, although Krul was not worked often enough in the Norwich goal.
When a second-half shot on target did finally arrive it was straight at the keeper, who did well to hold on to Emerson Palmieri's powerful strike at his near post.
Chelsea were denied a penalty when VAR ruled that Marco Stiepermann's challenge on Azpilicueta did not warrant further punishment, and the Blues' frustrations grew further when Barkley's 25-yard effort was comfortably kept out by Krul.
Lampard's side did finally make the breakthrough which their second-half dominance deserved with 22 minutes remaining when they hit Norwich on the counter-attack, the ball nestling into the hosts' net within 10 seconds of Kepa denying Pukki at the other end.
Questions marks could be raised over Krul, though, as he failed to get close to Abraham's long-range shot after the striker had cut inside Grant Hanley and let fly.
Norwich had offered very little going forward since the interval, but they almost equalised for a third time just four minutes after falling behind when Godfrey met a corner with a looping header which came back off the crossbar with Kepa rooted to the spot.
Godfrey was proving to be a thorn in Chelsea's side at both ends of the field as he made another important block to deny Barkley shortly afterwards, although the biggest grievance for Lampard may well be VAR.
The visitors had the ball in the back of the net again when Kurt Zouma converted from close range after Krul had spilled it at the feet of the defender inside the six-yard box, only for the video assistant to disallow the goal for a foul on the keeper by Olivier Giroud.
In a match which is sure to spark further debate over VAR another controversial decision went against Chelsea shortly afterwards when Godfrey appeared to stamp on the foot of Mount but was spared further punishment amid checks over a possible red card.
The decisions did not prove costly in the end, though, and it was Chelsea who looked most likely to add the game's sixth goal in the closing stages when Barkley drew a smart low save from Krul at the near post.
Lampard will have been happy enough with the one-goal winning margin as he finally got off the mark as Chelsea boss, although questions over his side's defensive form will remain after taking their tally of goals conceded for the season to seven in just three league games.
NORWICH (4-2-3-1): Krul; Aarons, Hanley, Godfrey, Lewis; Leitner (McLean 81'), Trybull (Vrancic 81'); Buendia, Stiepermann (Srbeny 81'), Cantwell; Pukki
CHELSEA (4-3-3): Kepa; Azpilicueta, Christensen, Zouma, Emerson; Kovacic, Jorginho, Barkley; Pulisic (Willian 84'), Abraham (Giroud 75'), Mount (Alonso 91')body check tags ::