Some intricate passing from Swansea ended with Nathan Dyer finding Graham, who took a touch before calmly slotting the ball in past John Ruddy to open scoring against the run of play.
Norwich tried to reply, but Andrew Surman headed straight at Michel Vorm from six yards out after being fed by Pilkington.
The Swans almost punished Surman's miss just minutes later, but a last-ditch block from Elliot Ward prevented Graham getting his second of the match.
The Canaries' equaliser came just two minutes into the second half after Ward played the ball across goal, allowing Holt to head in for his first of the game.
Just three minutes later, Paul Lambert's men were ahead. Elliot Bennett's cut back found Pilkington, whose shot looked like it was going wide before it deflected off Neil Taylor and wrong-footed Vorm.
Their third came just after the hour mark when Bennett found Holt, who proceeded to smash his shot past Vorm.
Norwich nearly extended their lead even further through a spectacular effort from Kyle Naughton, but Vorm just got enough on it to tip it onto the crossbar.
Swansea continued to push for a goal back, and had a penalty shout against Naughton turned down when the referee judged the ball to have hit his face, not his arm.
Both keepers then made two impressive saves in quick succession to keep their sides in the game - first Vorm from Wes Hoolahan and then Ruddy from a Sigurdsson free kick.
Swansea gave themselves a glimmer of hope late on when Holt conceded a penalty for a foul on Ashley Williams. Graham stepped up and sent Ruddy the wrong way to make it 3-2.
Norwich managed to hold on, however, giving them a valuable away win.