Stuttgart almost levelled the sides in the 16th minute when Matthias Ostrzolek's error gifted the ball to Mohammed Abdellaoue who in turn found Alexandru Maxim. The Romanian worked space to shoot, but his effort was well saved by Marwin Hitz.
It could have been 3-0 shortly afterwards when Ulreich completely missed a cross, presenting a glorious chance for Jan Moravek, but the Czech sliced his shot wide.
Stuttgart responded to that let off immediately, winning a penalty when Paul Verhaegh brought down Maxim inside the box. Vedad Ibisevic stepped up to take it and sent the keeper the wrong way to pull a goal back for the visitors.
They almost levelled the sides just two minutes later when Holzhauser's corner caused havoc in the area before coming back off the post.
The visitors' hopes of pulling off a comeback took a blow with 20 minutes of the match left when Ibrahima Traore was shown a straight red card against his former club having only been on the pitch for six minutes.
Sascha Molders should have wrapped the points up for the hosts in the 80th minute having rounded the keeper after seeing his initial shot saved, but the striker somehow fired into the side netting with the goal gaping.