Match winner Nikola Katic thanked Allan McGregor for the spot-kick save which spared him being tagged Rangers' penalty villain.
The big Croatian powered home a second-half header to seal Rangers' first Celtic Park victory in almost a decade.
But it could have been all so different for the Gers' centre-back had Ryan Christie slotted home the penalty Katic gave away with the game goalless for a blatant tug on Christopher Jullien's jersey.
McGregor, however, came to the rescue as he pushed away Christie's effort.
Celtic did find the net when a controversial Odsonne Edouard goal cancelled out Ryan Kent's opener but that merely set the stage for Katic to make his winning contribution.
He said: "It feels amazing. To come here and win in this place and make the gap between the sides really small was just amazing.
"I am really proud of my teammates and so happy for the fans.
"The goal was probably the best moment of my career until now. It's something we practise so much, we do it every single day and I was so happy to get the reward because we work so hard.
"Was I worried about the penalty? Almost. It was the referee's decision and maybe yes, maybe no, but Allan McGregor saved me and he saved the team."
Rangers have cut Celtic's lead at the top to two points and have a game in hand which could yet see them hit the front.
"We are in a good position but it's still winter," said Katic. "It's not even close to the end. We still have to take it game by game, session by session, keep pushing and giving everything to win each game."
Celtic skipper Scott Brown confessed Gers deserved their win but promised the Parkhead support his team will respond after the winter break.
"For us, it's always tough to take a defeat," he said. "We're in the same situation we were in last year when we lost the game at Ibrox.
"It wasn't our best performance today, we know that, but the main thing is that we're still top of the league and we've had a great run so far.
"Now, similar to last year, we have to make sure we bounce back after going to Dubai and having a good pre-season, coming back here fitter and stronger.
"We need to learn now, because that's two games in a row now where they've probably outplayed us.
"The main thing was that we won the Betfred Cup final, we're still top of the league. They've got a game in hand, but we're certainly not in too bad a place.
"It's the performances against Rangers that we need to make sure we improve on.
"For us, we know what we need to do and what we need to improve on.
"We'll have a few days off before meeting up in Dubai and we'll figure out how we're going to play against Rangers the next time. We'll bounce back."