Javi Martinez was sent off for conceding the stoppage-time penalty and Bensebain made no mistake from 12 yards.
Muller tried his luck again moments later, but again his effort fell short as the ball swung wide of the net.
Robert Lewandowski chipped inches wide of the post before returning for a second effort that again missed the target.
Muller was then thwarted for the third time when Sommer blocked a goal-bound shot and that near-miss was followed by another as Joshua Kimmich's shot was stopped millimetres short of the line and no goal was awarded.
Having dominated the first half, Bayern broke the deadlock after the break as Perisic cut inside and found Muller, who returned the ball to him before the Croatian winger volleyed into the top corner.
Gladbach were swift in their response, levelling when Jonas Hofmann's corner found Bensebaini, who delivered a powerful header.
Stindl's shot curled just wide of the post and Embolo was unable to get a foot to it as it flashed across goal.
A raft of bookings followed as both teams became frustrated by the stalemate, and crucially Martinez collected two in eight minutes and was dismissed – the latter for a trip that handed the home side the decisive penalty.
Bensebaini stepped forward to convert the resulting spot-kick to seal victory for Gladbach.