Wednesday's qualifying round second-leg tie at Celtic Park got off to a slow start as the visitors weighed in with a number of physical challenges, the first concrete chance not coming until the 23rd minute when Scott Brown fired just wide from the edge of the box following a corner.
Stuart Armstrong then tested Nenad Eric in the 42nd minute, before the Hoops were given the chance to take the lead from the spot when Abzal Beysebekov's foul in the box led to referee Istvan Kovacs blowing for a penalty.
Leigh Griffiths stepped up and rattled a shot inside the right post to put the hosts in the lead on the night and on aggregate.
Brendan Rodgers's charges, however, were caught out in the second half when, just a few minutes after Roger Canas's goal was ruled out for offside, Agim Ibraimi spotted Celtic goalkeeper Craig Gordon his line and lashed a shot into the net from around 40 yards out.
Just as it seemed like the game would go to extra time, Igor Shitov's foul in the area led to a second Celtic penalty in the game, as well as a red card for the Belarusian defender.
Dembele opted to take the spot kick himself and coolly slotted the ball inside the left post to send Celtic Park wild.