Les Bleus led for 64 minutes of the contest after two tries from winger Teddy Thomas, but Leidlaw, who switched to fly-half midway through the game, led the Scottish comeback with a series of late penalty kicks.
France got the first try of the game through Thomas. The back-rower picked up a loose pass sent wide, before stepping Scotland fly-half Finn Russell, holding off a challenge from full-back Stuart Hogg and crossing the line.
The French extended their advantage from a second successive penalty kick, but Scotland hit back through Sean Maitland. Russell responded to his earlier error by passing out wide for Maitland to touch down in the corner.
Thomas then grabbed his second try of the game to extend France's lead, chipping the ball before reaping the benefits of the bounce and racing clear.
Laidlaw could have reduced the arrears after sprinting clear from an attack, but the scrum-half knocked on close to the line. The hosts received the ball back from a French mistake, however, before Laidlaw recovered to set up Huw Jones for a try.
Franc went into half time with a 20-14 lead, after Maxime Machenaud, who had converted France's earlier tries, slotted a third penalty in quick succession for the away side.
A scrappy start to the second half saw penalties exchanged, but Scotland eventually got level with their third penalty of the half through Laidlaw.
The hosts then pressed on, converting two more penalty kicks and securing their first win in this year's Six Nations.