© PA Photos
Celtic's first sight on goal came after 16 minutes when Emilio Izaguirre sent a cross into the box, but Miku could not get on the end of it.
Joe Ledley missed a great chance on 27 minutes, putting his shot wide of the goal.
Cillian Sheridan came close from a corner, but Fraser Forster cleared it away from his goal.
Sheridan put Kilmarnock ahead two minutes before half-time, taking the ball around Forster in a one-on-one before slotting home.
After 62 minutes, Kilmarnock were awarded a penalty and Liam Kelly converted to double the away side's lead.
Izaguirre pulled a goal back for Celtic on 76 minutes, but his goal was ruled out for offside.
Celtic threw everything at Kilmarnock, but could not find a way through.
Cammy Bell then denied Tony Watt from inside the penalty box after 86 minutes.
The final whistle blew with Kilmarnock securing all three points at Parkhead.