An excellent free kick from Anthony Stokes won it for the champions, who had been frustrated for much of the game.
The hosts were able to get back into the match before half time, but neither side were able to create anything clear-cut.
Just as the game seemed to be heading for a goalless draw, Stokes broke forward and skipped past a couple of challenges before being brought down on the edge of the box.
The Celtic striker picked himself up and stepped forward to curl an excellent effort in off the post.
The visitors could have added to their lead as they pushed forward on the break in the final minutes, but it did not matter as Stokes's 50th Celtic goal secured all three points.