Harlequins replied through the boot of Ben Botica, but two tries in the space of five minutes helped the hosts establish a clear advantage at 18-6.
French international centre Wesley Fofana touched down midway through the first period after running onto a pass from James, who added a conversation to Thierry Lacrampe's effort just five minutes later.
A try on the stroke of the break from Tom Williams, following a powerful charge from the visiting forwards, brought Harlequins within touching distance, with the scores at 20-13 at the interval.
They could not mount a second-half comeback, though, with the points being limited to three for each side, James successfully kicking a penalty that was cancelled out by Evans's late drop goal.
This was Clermont's first win of this year's competition, while Harlequins have now lost both of their opening two group-stage matches.