Despite dominating possession for much of the match, Stuttgart found it difficult to slice open the home defence and Kiel, who play in the third tier of German football, wasted a rare opportunity early on when Marc Heider headed straight at Przemyslaw Tyton.
However, an upset appeared to be on the cards when Rafael Czichos found space inside the box to pick out the bottom corner with a powerful strike.
The lead lasted just four minutes, though, as Daniel Didavi linked up with Ginczek in the box before lobbing Niklas Jakusch.
Stuttgart started the second half on the front foot, with Didavi coming close to a second goal when his curled effort was pushed clear by the Kiel shot-stopper.
An impressive defensive display had raised hope of a shock victory for the home side in the first hour, but a lapse of concentration gifted Stuttgart the lead when Ginczek found space in the box to convert a Timo Werner cross.
Kiel had chances to force extra time as Hauke Wahl headed wide, before Heider's low strike was kept out by Tyton as Stuttgart held on to reach the second round.