Blades boss Chris Wilder responded to growing speculation over his future by making only one change from the side beaten by Crystal Palace, although injury and Covid-19 had reduced his options.
Wilder named eight substitutes despite being allowed nine on the bench, but the visitors were out of the blocks fast and Day saved from Ben Osborn with Lys Mousset almost bundling in the rebound.
Mousset had been reminded about his responsibilities following an incident last weekend when images showing a smashed Lamborghini belonging to the French forward circulated on social media.
But Mousset was busy making positive headlines when meeting Osborn's sixth-minute cross with a fine header. Day got the faintest touch on to the crossbar, but was helpless as the ball struck him on the back for the most unfortunate of own goals.
Rovers were level with their first serious attack after 21 minutes as Kilgour met Zain Westbrooke's free-kick with a towering header from 12 yards.
Rovers survived a penalty appeal when Luke Leahy and Bogle tangled and Day, part of Newport's run to the fifth round two years ago, denied Lundstram from close range.
The ball was soon in the home net again when Mousset tucked away a rebound after Day pushed out John Egan's effort.
Referee Stroud came to Rovers' rescue again by insisting Egan had climbed over Leahy to win the header.
Burke replaced Mousset during the early stages of the second half and the Scotland international should have scored with his first touch.
But Burke sliced Osborn's cross horribly wide before McGoldrick and Lundstram went close as the pressure began to grow on the Rovers goal.
Burke made amends for his earlier miss after 59 minutes, striding on to Oliver Norwood's measured pass as Rovers' complaints of a free-kick in their favour at the other end fell on deaf ears.
Rovers were level within three minutes as Jack Baldwin flicked on a corner to the far post where the unmarked Ehmer finished.
But parity lasted less than 60 seconds as Bogle started a move involving Burke and McGoldrick, and he maintained his run to stroke home his second Blades goal and ensure Wilder's men had the winning feeling at last.