The League One hosts held their own for long spells of the contest, but goals from Marco Matias and Adam Reach ensured that Wednesday progress into the next round, where they will host Premier League outfit Wolverhampton Wanderers.
The Owls got off to a slow start in front of a sparse home crowd in the North-East, and that seemed to encourage Sunderland as they looked to continue their unbeaten start to the season.
It took until the 19th minute for the first chance of any note to arrive, though, and Elliot Embleton could not make the most of it as he dragged his effort wide of the target.
Wednesday responded immediately to that, though, and Matias offered a warning shot of his imminent opener just seconds later when he drew the first save from Jonathan McLaughlin after latching on to Steven Fletcher's knockdown.
Matias still needed a sizeable helping hand when he did break the deadlock shortly before the half-hour mark, though, as Alim Ozturk made a complete mess of shepherding the ball back to his keeper, instead teeing up the Wednesday attacker, who poked a simple finish past McLaughlin.
Sunderland responded well to falling behind, but clear chances were still at a premium for the hosts and Luke O'Nien saw arguably the best of them go begging when his close-range strike was blocked by Joost van Aken.
The second half got off to a more open start with sights of goal at both ends, and Matias came closest 10 minutes after the restart when his strike was cleared off the line by Jack Baldwin with the keeper beaten.
Wednesday were enjoying a growing degree of control over the game and looked increasingly likely to get a second goal, with McLaughlin needing to be alert to deny Reach and Tom Lees glancing a header wide from Barry Bannan's delivery just after the hour mark.
Fletcher also drew a save from McLaughlin before the second goal did finally arrive through Newcastle United supporter Reach, who glanced Bannan's pinpoint delivery past the keeper and in off the far post.
Sunderland offered little sign of pulling a goal back in the final 10 minutes as Wednesday comfortably saw out the win, recording back-to-back away victories over the Black Cats for the first time since 1914 in the process.
SUNDERLAND (4-4-1-1): McLaughlin; Hume, Ozturk, Baldwin, James (Molyneux 80'); O'Nien, Cattermole (Maja 66'), Power, Embleton (Gooch 61'); Honeyman; Maguire
SHEFFIELD WEDNESDAY (3-4-3): Wildsmith; Fisker, Lees, Van Aken (Penney 63'); Palmer, Hunt (Kirby 46'), Bannan (Lee 86'); Boyd; Matias, Fletcher, Reach