Chorley, of the National League North, overcame the League One side, winners of the competition as recently as 2013, 3-2 after extra time on Sunday and will now travel to London Road to face Posh – the highest ranked team in this stage of the competition.
Eighth-tier sides Canvey Island, the lowest ranked club still involved, and Marine both have home ties against Boreham Wood and Havant and Waterlooville respectively.
Kings Lynn, in the second round for the first time in their history, will travel to 2008 winners Portsmouth.
Crawley, who won an astonishing game 6-5 at Torquay in the first round, will play away at either Barrow or AFC Wimbledon, with that tie still to be played due to Covid-19 issues.
Brackley Town have never been past round two and they will have to overcome Tranmere at Prenton Park if they are to make a piece of club history.
Solihull Moors knocked out Scunthorpe at the weekend and will look to claim another League Two scalp in the shape of Morecambe, while Stockport, who upset Rochdale, will host Yeovil.
League Two leaders Newport were handed a home draw against Salford, while Darlington will travel to Bristol Rovers.