Goals early in each half ensured the Championship side were untroubled by their League Two hosts, although debutant Luke Norris missed a late penalty which could have set up an exciting finale.
Swansea lost 5-1 at QPR at this stage last season but there was to be no repeat this time around.
Routledge headed in Yan Dhanda's seventh-minute cross to score 10 years and a day after he had played in Premier League side Newcastle's FA Cup loss at the same ground.
Stevenage started the second half well but were caught out five minutes after the break with Gyokeres' glancing header from Jordan Garrick's cross finding the net.
In foggy conditions, the hosts should have pulled one back nine minutes from the end as Gyokeres handled in the area but former Colchester striker Norris' spot-kick was brilliantly saved by Freddie Woodman.