Striker Gary McSheffrey gave Coventry the lead after only five minutes with a well-struck shot into the roof of the net.
Richard Wood had the opportunity to double his side's advantage but his header cleared the crossbar.
Southampton nearly equalised through Jack Cork but the midfielder's shot just went wide of the post.
The away side improved in the second half and James Ward-Prowse equalised for Southampton in the 63rd minute from close range after a cross from Daniel Fox.
More good work from Fox presented an opportunity for Jos Hooiveld but he failed to beat Chris Dunn in the Coventry goal.
Dunn quickley turned from hero to villain as Aaron Martin headed a Fox corner through the keeper's hands with eight minutes of normal time remaining to win the tie for Southampton.