The league's bottom side got off to the worst possible start, after Steven Thompson sent the Saints into the lead with just one minute on the clock.
Moments later, David Smith found Callum Patterson from the right, with the striker steering his header past former Hearts keeper Marian Kello to bring the scores level with five minutes played.
The visitors almost regained their lead, though Dylan McGowan's last-ditch tackle on Thompson from six yards out kept the match tied.
John McGinn thought he had fired Danny Lennon's side back into the lead, but his header came back off the crossbar to the relief of the home supporters.
After avoiding falling behind for a second time, Hearts attacked up the other end and were rewarded when Dale Carrick won his side a penalty following a Marc McAusland foul which earned the defender a straight red card.
Jamie Hamil made no mistake of converting the penalty, calmly slotting into the bottom right-hand corner to give the League Cup semi-finalists the lead.
Gary Locke's men very nearly made it three after David Smith's ball from the right was flicked on by Carrick before ending up inches wide of the far post to keep the score at 2-1.
Hearts defended for their lives in the closing stages, but held on to pick up their second successive victory in the league.