Neither side were in control of the game in the opening stages, but the visitors opened the scoring with the first real chance of the game on nine minutes.
Arango and Raffael then had chances before the half-hour mark to double Gladbach's advantage, but both players put their chances wide of the target.
Oliver Sorg then brought down Raffael in the box, but Filip Daems's penalty was turned around the post by Oliver Baumann.
Then the game turned around in the 70th minute. Granit Xhaka was sent off for his second yellow card, then Sorg rifled a shot from range into the net for a dramatic turnaround.
Gladbach chased the game but they were undone twice more by Vladimir Darida and then Mehmedi, before they grabbed a late consolation through Havard Nordtveid.