Jan Bednarek’s header and a fine James Ward-Prowse free-kick had Saints dreaming of a first home league win against the Red Devils since 2003.
But Ralph Hasenhuttl’s men squandered their two-goal lead as half-time introduction Cavani turned the match on its head, setting up Bruno Fernandes before netting a brace to secure a sensational late triumph.
It was the 10th time that United had come from behind to beat Southampton in Premier League history – a victory that propelled them up the standings and continued their 100 per cent start to life on the road.
Few could have foreseen this outcome when Ward-Prowse superbly beat David De Gea shortly after sending in a smart corner for Bednarek to open the scoring at an empty St Mary’s.
Solskjaer turned to summer signing Cavani at the break and could not have dreamed of a better outcome as he provided the assist for Fernandes before clinching victory with a brace.
It was a wild end to a match that United had started on top before things went awry.
Saints were unsettled by the visitors’ confident start and were breathing an early sigh of relief as a misunderstanding at the back allowed returning Mason Greenwood to beat Alex McCarthy but hit the side netting.
Harry Maguire fumed at referee Jon Moss after Marcus Rashford went down in the box under pressure from Ward-Prowse, with Fernandes seeing a deflected effort from distance whizz just wide.
United were causing the hosts all sorts of problems, but the momentum turned in the 23rd minute.
Ward-Prowse fired an inswinging corner to the near post, where Bednarek had wriggled away from Rashford and glanced a header home from close range.
The opener rattled United and emboldened Saints, with Ward-Prowse’s corners to the near post continuing to cause havoc and Kyle Walker-Peters seeing a deflected shot hit the far post.
Hasenhuttl’s side were now on top but McCarthy almost threw away that building momentum when inexplicably passing straight into the path of Greenwood. The Southampton goalkeeper saved the teenager’s attempt to capitalise, then superbly spread himself to stop Fernandes’ follow-up.
That let-off allowed Saints to land another gut punch. Fred’s foul on Moussa Djenepo gave Ward-Prowse a tantalisingly-placed free-kick that was duly dispatched, with De Gea just unable to prevent the curling right-footed effort from finding the top corner.
Stuart Armstrong fluffed his lines after De Gea denied Djenepo as Southampton pushed for a third before half-time, which the United goalkeeper did not return from due to picking up a knock.
Theo Walcott was quick to test back-up goalkeeper Dean Henderson in the early minutes of a second half that saw United threaten.
McCarthy did brilliantly to deny Rashford when sent scampering behind the Southampton backline. Cavani looked well-placed if the forward had passed instead, with the substitute soon rifling over on the turn.
United’s building pressure would tell as Cavani lifted the ball across to Fernandes, who struck low past McCarthy after taking a touch.
The visitors ratcheted up the pressure as they looked to take something from St Mary’s.
Cavani was proving a handful and levelled proceedings in the 74th minute. Saints goalkeeper McCarthy punched a corner clear and Fernandes’ deflected shot back into the box was headed home by poacher Cavani.
Che Adams fired across the face of goal on a rare voyage forwards by Southampton, who were under the cosh and praying for the final whistle.
But just as it looked like they had held out, United dug deep to break the hosts’ hearts in stoppage time. Cavani peeled away to meet a Rashford cross and beat McCarthy to the delight of those connected to the visitors.