The home side struggled to create much early on as the game was broken up by two lengthy injury stoppages.
Lee made amends 13 minutes later as he put the away side ahead with a powerful header from Lewis Buxton's cross.
Elphick went close with a second header shortly after the break, before Wednesday almost doubled their lead as Chris Maguire struck the post with a fierce free kick.
The home side were reduced to 10 men shortly after, though, as Simon Francis was sent off for a second bookable offence.
However, they rallied and found a second goal as Ritchie fired the ball into the back of the net from a free kick.
Wednesday came within inches of equalising when Lavery stretched to get his toe to a low cross with the goal gaping, and they were awarded a penalty for a foul on Atdhe Nuhiu, which Maguire slotted deep into added time.