© PA Photos
Leeds started the stronger and Luke Varney spurned an early opportunity by missing the target with a close-range header in front of an open goal.
Varney continued to be the main attacking threat for the hosts, hammering the crossbar with a volley just after the 20-minute mark.
The hosts eventually took the lead when Varney's cross from the right fell to Michael Tonge on the edge of the area to tuck home.
Premier League side Southampton created very little on the night and conceded two late goals to compound their misery.
The draw for the fifth round takes place tomorrow night.