Celtic, who started the night trailing their opponents 2-0, nearly scored their first goal of the tie in the fifth minute when Mikael Lustig's header was tipped over the crossbar by Alexandr Mokin.
Karaganda struggled to contain the hosts in the early stages, with both Anthony Stokes and Charlie Mulgrew testing the busy Mokin before the game had reached the 10-minute mark.
It looked like it might be a frustrating evening for the Celtic supporters when a free kick from Stokes, which looked destined for the back of the net, was pushed to safety.
As the nerves grew with the contest approaching the interval, Kris Commons broke the deadlock in fantastic style when, after collecting a pass from Georgios Samaras, the attacking midfielder created space to fire in from 25 yards.
© Getty Images
Celtic continued to dominate proceedings early in the second half and their superiority brought a second goal as Samaras took advantage of some space to place a shot into the roof of the net and level the tie.
The hosts should have been ahead on the hour mark but Stokes somehow volleyed the ball against the crossbar from close range.
Karaganda nearly punished their opponents for wasting such a great opportunity when Celtic were forced to scramble the ball away after a goalmouth scramble ended with the crossbar being rattled.
The contest remained open as the final whistle approached and extra time was looming before one final twist in stoppage time.
Stokes beat a couple of defenders on the left side of the box before crossing for Forrest, who showed great composure to rifle the ball into the back of the net and send his team into the group stages.