The game started well for the English side, with Mark Cueto scoring a try after three minutes.
The remainder of the first half was a battle of the boot, with Montpellier awarded five penalties and Martin Bustos Moyano kicking four of them successfully.
Rob Millar kicked a further three points for Sharks but missed a second penalty, making it 12-8 to Montpellier at the break.
Millar brought Sale to within one point of the home side with a further penalty kick in the second half.
Richie Vernon gave Sale some late hope, running in a try with six minutes left to play, but the side couldn't build on the momentum.
Sale are second in pool six behind Toulon, remaining ahead of Montpellier on points difference.