Sweden's 3-0 victory over Mexico meant that Germany needed to record three points but late goals from Kim Young-gwon and Son Heung-min condemned the defending champions to their first early exit since 1938.
South Korea's heroics in added-on time does not secure qualification for the last 16, but it meant that Germany end their Group F campaign in bottom spot.
Both teams went into this contest knowing that victory could secure their place in the knockout stages but chances were few and far between during an opening 45 minutes which was largely played at a pedestrian pace.
The first real opening of the game went the way of South Korea as Manuel Neuer failed to keep hold of a long-range free kick from Jung Woo-young, but the Germany goalkeeper recovered to claw the ball away ahead of Son.
Germany were seeing more of the ball but Son got his chance soon afterwards after the ball was only half-cleared inside the penalty area, but the Tottenham Hotspur attacker volleyed off target from 14 yards out.
As the first half drew nearer to a close, Germany improved and Jo Hyeon-woo needed to be alert to smother a close-range effort from Mats Hummels, who had done well to create a chance for himself in the box.
However, Germany went into the break having once again under-performed in a first half at this tournament but they should have gone ahead soon after the restart when Joshua Kimmich crossed for Leon Goretzka, only for the midfielder to see his header tipped wide of the post by Jo.
At a time when news of Sweden's opener against Mexico began to filter through, Timo Werner was presented with the chance to get a half-volley away from by the penalty spot but the forward drilled his effort wide of Jo's right-hand post.
The match was becoming increasingly open but while both teams were getting on the ball in the final third, opportunities were few and far between with Mario Gomez wasting the best opening by heading straight at Jo.
As Sweden went three up in the other game, Germany became desperate to find the goal which would earn their progress and eliminate Mexico and Marco Reus almost broke the deadlock in some style with his 25-yard strike missing the far corner by a matter of inches.
With three minutes left, Hummels should have scored when meeting an inswinging cross but the centre-back somehow sent his free header off target from no more than eight yards.
South Korea had appeared finished from an offensive perspective but twice in six minutes of added-on time, Shin Tae-yong's side found a way through to condemn Germany to one of the darkest days in their history.
The assistant initially ruled out Kim's opener - which was netted from close range after a corner - but VAR showed that two Germany players had touched the ball before it had reached the defender, leaving the referee to reverse the initial decision.
Germany now needed two goals and the ball was delivered continuously into the penalty area but with goalkeeper Neuer left stranded after going up for a corner, a long ball was played through to Son who reached it inside the six-yard box before tapping into an empty net to cap a famous triumph for his nation.
SOUTH KOREA (4-4-2): Jo; Lee Wong, Yun, Kim, Hong; Lee Jae-sung, Jang, Jung, Moon (Ju 69'); Son, Koo (Hwang 56', Ko 79')
GERMANY (4-2-3-1): Neuer; Kimmich, Hummels, Sule, Hector (Brandt 78'); Khedira (Gomez 58'), Kroos; Ozil, Goretzka (Muller 63'), Reus; Werner