The Exiles fell behind to their Championship opponents early on in Sunday's third-round encounter at Rodney Parade, but were level on 76 minutes through Conor Shaughnessy's own goal.
Just when it seemed like the tie would be going to a replay at Elland Road, McCoulsky rose high in the box to nod home a corner kick and secure a 2-1 win over the Whites.
"This is much better beating Championship teams like Leeds," Flynn told BBC Sport. "I don't want to be winning relegation battles, I want to be winning FA Cup ties and climbing the table.
"I thought we were the better team in the first half, the goal was unfortunate, but we had chances and kept going. I told them we don't have to win the game in the first 10 minutes of the second half and we got crosses into the box.
"It was an outstanding header. He has 11 goals for us this season and he has been superb for us. He was hungry and wanted it and I am delighted."
The result ensures that Newport are into the fourth round of the FA Cup for the first time since 1979, with the draw taking place on Monday evening.