Geddes put his side ahead in the first half from the penalty spot after Coventry goalkeeper Lee Burge had been sent off.
Worcester's Nathan Vaughan then stepped up to save Reda Johnson's penalty following Graham Hutchinson's foul on James Maddison before Geddes doubled his side's lead with a rifled finish from the edge of the area.
Johnson pulled a goal back for Coventry inside the last ten minutes, but Worcester held on to book their place in the second round.
However, Norton United, making their first ever appearance in the FA Cup first round, were beaten 4-0 by Conference side Gateshead thanks to a hat-trick from Rob Ramshaw and an effort from Alex Rodman.
Chesterfield strolled to a 6-0 victory against Braintree Town thanks to two goals from Eoin Doyle, efforts from James O'Shea, Sam Clucas, a Remy Clerima own goal and a stunning 20-yard volley from Gary Roberts.
There was no upset at New Lawn as a brace of deflected strikes from Gary McSheffrey gave Scunthorpe United a 2-0 win over Forest Green Rovers, while Wrexham strolled to a 3-0 win over Woking in the all-Conference clash.
Meanwhile the clash between League Two Stevenage and Maidstone United of the Isthmian Premier Division was one of two ties to finish goalless, alongside the clash between Notts County and Accrington Stanley.
Results in full: Blyth Spartans 4-1 Altrincham, Stevenage 0-0 Maidstone United, Forest Green Rovers 0-2 Scunthorpe United, Braintree Town 0-6 Chesterfield, Gosport Borough 3-6 Colchester United, Norton United 0-4 Gateshead, Coventry City 1-2 Worcester City, Portsmouth 2-2 Aldershot Town, Wrexham 3-0 Woking, Notts County 0-0 Accrington Stanley