The home side looked lively early on and in particular left-back Constant Djakpa. He was causing his man problems with his runs and could have scored inside five minutes, but his low shot was kept out by Loris Karius.
The game became a little bit scrappy as Mainz continued to struggle for chances, with Zdenek Pospech summing up their opening 20 minutes by shooting well over the bar.
Joselu then had a great chance to score, but his header was clawed away before he was only able to turn the rebound wide of the near post.
Just after the break, however, the Spaniard did give his side the lead, which was just about deserved, with a towering header from inside the box.
Alexander Meier then sealed the result late on with five minutes to play with a fine strike.