Both sides had a series of chances as the half-hour mark approached, but Josh Magennis and academy player Aaron Splaine both saw their efforts blocked, while Jamie Reckford and Scott Boyd were both denied by Conor Brennan down the other end.
It was Kilmarnock who broke the deadlock less than 10 minute before the break when Magennis got forward again, squaring the ball to Greg Kiltie to leave striker to slot it home.
Splaine could have doubled the home side's lead just a couple of minutes later as he tried his luck with a low strike from outside the box, but a diving stop from Antonio Reguero denied the midfielder.
County pushed for an equaliser before the break, but Martin Woods's driven effort was blocked before Gardyne's finish was saved after a quick rebound.
The visitors were still on the attack at the start of the second half as Liam Boyce attempted to catch Brennan out with a powerful strike towards the top corner.
Their persistence paid off with just over an hour gone as Marcus Fraser curled in a corner, leaving Boyce to smash it into the back of the net from close range.
The visitors took the lead just over 10 minutes later as Boyce provided a through ball for Rocco Quinn, who converted simple finish into the bottom-right corner.
The win lifts Ross County above Kilmarnock into ninth in the table, one place above their opponents.