The defender's back post header glanced off the post before nestling in the net after Kerr connected with Danny Swanson's cross, sealing a 2-2 draw for the visitors at Easter Road.
However, Florian Kamberi restored the home side's lead within 60 seconds, netting his fifth of the campaign – and it looked like bringing three points until Kerr's late intervention.
The hosts started on the front foot as they attempted to worsen Saints' early season woes.
Kamberi went even closer three minutes later when his effort struck the post.
Saints weathered the early storm and started to get to grips with Paul Heckingbottom's side.
But just as Saints were beginning to look more menacing in the final third, the home side broke the deadlock, Jackson climbing unchallenged inside a congested box to nod Stevie Mallan's free-kick back across goal and into the net.
Hibs goalkeeper Ofir Marciano heaved a sigh of relief within four minutes of the restart that he only received a booking for fouling O'Halloran outside his box.
The Israeli tripped the winger by the side of the area as the pair chased down Kerr's pass.
There was more injury woe for Hibs in the 56th minute when captain David Gray was forced off following an accidental collision with Saints left-back Tanser.
Saints had the next chance two minutes later as Marciano had to stretch to parry Liam Craig's shot from distance.
At the other went, Allan found Doidge in the box with a deft pass but the marksman's tame left-footed effort was easily collected by Zander Clark.
Boos rang around Easter Road on the hour as the home fans made their feelings clear in the decision to replace Allan, the midfielder making way for Daryl Horgan – and there were more jeers when Saints levelled midway through the second half.
Craig found O'Halloran inside the area with a great reverse pass and the midfielder caught out Marciano at the goalkeeper's near post with a drive high into the net.
Parity lasted only 60 seconds as Hibs immediately responded. Glenn Middleton's low cross came back off Kamberi and spun past the hapless Clark.
However, Saints piled on the pressure and made it count in the fourth minute of added time as Kerr rose high in the box to head home.