Danny Swanson's 29th-minute goal gave Peterborough the lead at half time.
Watford responded well and Almen Abdi tested Robert Olejnik in the Peterborough goal just before the break.
Manager Gianfranco Zola brought Matej Vydra on at half time, as his side switched to a 4-3-3 formation in search of an equaliser.
However, it was Peterborough who went two goals in front on the hour mark after Tomlin squared the ball for Gayle to tap into an empty net.
Gayle continued to cause problems for the Hornets, and Marco Cassetti was shown a straight red card for bringing down the Peterborough forward on 65 minutes. From the resulting free kick, Tomlin curled beyond the reach of Manuel Almunia to seal all three points for the home side.
Late consolation goals from Mark Yeates and Fernando Forestieri made for a tense finale, but with Hull City winning away at Ipswich Town, Watford are now four points adrift of the second-placed Tigers with just four games left to play.