Nathan Austin put the home side back in front with 30 minutes left, with it remaining that way until the final whistle.
East Stirlingshire, meanwhile, secured a 3-1 win at home to Annan Athletic, despite Nigerian forward Smart Osadolor putting the visitors in the lead shortly after the 30-minute mark. Kris Faulds equalised for the hosts a few minutes before half time, before Nathan Shepherd scored two goals in 16 minutes to put the game beyond Annan.
Willie Robertson pulled one back shortly after to give the Berwick a strand of hope, but it finished in defeat for the home side.
Arbroath remain bottom of the table after losing to Queen's Park, who went ahead within the opening five minutes thanks to a Paul Woods goal, and doubled their lead three minutes into the second half with a Sean Burns finish. The home side responded in the 90th minute through Andy Stirling, but it was nothing more than a consolation goal.