Schalke were on top from the first whistle, but they were forced to wait for their first clear opportunity as a determined visiting defence limited the space on offer.
© Getty Images
Despite offering very little going forward before the break, it was Koln who broke the deadlock in the first minute of the second half as Yannick Gerhardt released Anthony Ujah, who kept his composure in the box to slot past Ralf Fahrmann.
Koln could have been further ahead just a few minutes later, but Fahrmann was off his line quickly to deny Dusan Svento after the midfielder had timed his run forward to perfection.
Matthias Lehmann stepped up from 12 yards to beat Fahrmann with an emphatic finish and give Koln a crucial two-goal advantage.
The penalty proved to be the turning point as Koln finished the game confidently, with Ujah firing straight at Fahrmann late on.
Schalke pulled a goal back in the 85th minute when substitute Leroy Sane converted from an Atsuto Uchida cross, but the visitors held on for all three points.