The former Manchester City striker opened the scoring moments before the break by heading in a cross from Cha Du-Ri.
He then doubled the lead eight minutes after the restart when he scored from the edge of the area following a pass from strike partner Gary Hooper.
Celtic's advantage was halved four minutes from time when Killie's Danny Racchi shot into the bottom left-hand corner of the net from distance.
However, Racchi's goalscoring intervention failed to spur the visiting side to find an equaliser.
Celtic take on Rangers next week.