The Whites took the lead after just nine minutes when Gaetano Berardi fired a low effort through a crowd of bodies and into the back of the Exiles' net from 30 yards for his first Leeds goal.
With just one minute of normal time remaining, Robbie Willmott's pinpoint corner kick from the left fell to substitute Shawn McCoulskey, who rose highest to power home a header and send the crowd wild.
The afternoon would get even worse for Leeds as substitute Samuel Saiz got himself sent off for allegedly spitting at Willmott in stoppage time, culminating in a brawl between both sets of players in the centre circle.
The result ensures that Newport are into the fourth round of the FA Cup for the first time since 1979, with the draw to take place on Monday evening.
NEWPORT COUNTY (4-5-1): Day; Pipe, White, Demetriou, Butler; Willmott, Bennett, Labadie, Dolan, Nouble; Amond
LEEDS UNITED (4-4-2): Lonergan; Anita, Shaughnessy, Berardi, Borthwick-Jackson; Sacko, Phillips, Klich, Cibicki; Grot, Lasogga