The Tottenham Hotspur winger scored twice in the final 10 minutes to cancel out James Morrison's opener and earn his country their first points of the campaign.
Both sides were involved in a frantic start to the match as Bale fired well over from 25 yards.
The hosts should have taken the lead soon after but Steve Morison somehow put his header wide from Bale's cross.
They were made to pay on 26 minutes as Morrison latched onto Steven Fletcher's flick-on before firing powerfully past Lewis Price.
The home team refused to get their heads down despite going behind and pushed hard for an equaliser. However, Joe Allen and Aaron Ramsey both put their efforts wide as the Scots held onto their lead at the break.
Wales were dominant in the second half and had a host of chances to get themselves level.
They were appealing for a penalty midway through the half when Ramsey looked to have been brought down by Charlie Adam, but the referee waved away the protests.
Ramsey, who will miss his side's next qualifier after picking up a booking, then fired wide.
Bale continuously threatened the Scotland goal and went close on 70 minutes when he fizzed his effort off target.
Scotland were inexplicably denied a goal with 15 minutes to go when Steven Fletcher's header was chalked off after the linesman had ruled that Charlie Adam's cross had swung out of play.
Chris Coleman's team got a deserved equaliser with 10 minutes left to play thanks to Bale's penalty following a foul on the winger by Shaun Maloney.
Bale then fired in a spectacular winner from 25 yards with just minutes remaining on the clock to earn the Welsh their first victory of the campaign.