Jermaine McGillvary went over in the corner early on for the visitors, but the try was ruled out by the video referee after replays showed that his foot was in touch.
From the restart Quinlan picked up James Roby's deflected kick and crashed over to put St Helens ahead, with Walsh adding the extra points.
The lead was short lived as McGillvary gathered Scott Grix's grubber and raced clear to make it 12-12, but not to be outdone, Quinlan then sealed his first-half hat-trick to make it 18-12 at the break.
Just as it seemed that St Helens might put some distance between themselves and their guests, McGillvary scored his second of the game to level things up once more at 24-24.
However, St Helens were undeterred, and took control of the game when Joe Greenwood and Percival went over in quick succession to open up a 10-point lead.
There was more drama though as Eorl Crabtree set up a grandstand finish when he reduced the deficit to just four points on 68 minutes, and when Aaron Murphy went over three minutes later it appeared the Giants might snatch a remarkable win.
In another twist, Jamie Ellis missed the kick to leave the game tied at 34-34 with five minutes to play and allow Walsh to pop up with a drop goal inside the final two minutes to win the game by a single point for St Helens.