The former Killie striker pounced in the second minute of time added on after coming off the bench to replace Brian Graham midway through the second half.
His initial shot from the edge of the box smacked off the crossbar and rebounded off goalkeeper Laurentiu Branescu back into his path and Erwin duly smashed it home from 12 yards.
The first half was conducted at a frenetic pace, with play swinging from end to end once County had got to grips with the high tempo.
But the substitution did not seem to bother the home side and Graham threatened the Killie goal before Eamonn Brophy replied in kind for the opposition.
County enjoyed a period of supremacy half-an-hour in and had three chances to score in as many minutes.
Stewart leaned too far back and blasted a shot over the crossbar before Mullin drove his attempt into the defensive wall after McKay had rolled a free kick to him.
Chris Burke alleviated the pressure in the 35th minute when he conjured up a superbly-struck free-kick from 25 yards and skimmed the crossbar with his effort.
Branescu then had to sprint from his goal to beat Stewart to the ball when County launched a counter-attack immediately prior to Brophy shooting inches over.
Perhaps not surprisingly, the second half lacked the intensity of the first 45 minutes, but there was still enough to keep the 3,688 crowd entertained.
But the defender fluffed his headed finish before Brophy tested on-loan Chelsea goalkeeper Nathan Baxter a short time later.
Had Erwin found the net with his spectacular effort on the run in the 77th minute it would have been a candidate for goal of the month, but the ball curled a foot or so wide of the far post.
County were immediately inspired and Fontaine and Mullin went desperately close to scoring.