Japan had their eyes on revenge after losing to the Scots at last year's Rugby World Cup, but it was ultimately a comfortable victory for the visitors, who put on a strong second-half performance.
Scotland skipper Greig Laidlaw sent the ball between the sticks to give his team a sixth-minute lead but Japan soon replied with a try from Shota Horie, who capitalised on a missed tackle from Duncan Taylor to go over the line.
© Getty Images
Two penalties from Laidlaw regained Scotland the lead but a kick from Yu Tamara ensured that Japan held a narrow lead on the half-hour mark.
However, just before the break, Scotland were awarded a penalty try after an infringement from Rikiya Matsuda and after Laidlaw's conversion, Scotland were 16-10 ahead at the interval.
Japan's ill-discipline had seen them reduced to 13 men and Scotland took full advantage when WP Nel powered through three challenges to go over the line, before Laidlaw converted for the second time in the match.
A second penalty from Tamura moved Japan onto 13 points in the 57th minute, but Laidlaw responded with a penalty of his own seven minutes later to make it a 13-point victory.
The two teams will meet again at Tokyo's Ajinomoto Stadium for a second Test on June 25.