The League One side secured a memorable meeting with the unbeaten Premier League leaders thanks to Pierre's low finish from outside the area in the 89th minute.
Shrewsbury defender Pierre's long-range effort was just off target and home skipper Ollie Norburn's shot was deflected narrowly wide.
Shrewsbury striker Daniel Udoh forced a fine save from Daniel Bentley at the other end with a powerful effort from just outside the area.
Diedhiou fired wide for City early in the second half before home substitute Callum Lang's header was well kept out by Bentley.
With extra-time looming, Shrewsbury delivered the knockout blow in the 89th minute as Pierre picked out the bottom corner to allow his side to dream of an even bigger upset.