The visitors applied plenty of pressure on the Berwick defence in the opening minutes with Youssef Bejaoui having to be on alert on a number of occasions in goal for the hosts.
Emilson Cribari should have put Ally McCoist's men ahead midway through the first half but his attempted back-heel trickled safely past the post.
Berwick looked to have survived the first period until deep into stoppage time when Andrew Little fired a shot powerfully into the roof of the net.
The home side started much better at the beginning of the second half, with Rangers being forced to stave off a number of early attacks.
Rangers were shocked in the 62nd minute as substitute Fraser McLaren managed to pull a goal back to the delight of the Sheffield Park crowd.
Lee McCulloch almost immediately restored his team's lead but the midfielder's effort from outside the area flew just wide of the post.
As Rangers searched for a winner they could have found themselves behind when McLaren collected the ball inside the box but the striker's curling effort was wayward.
In the final minutes, the hosts finished the better side with McLaren yet again being denied a second goal by a superb Neil Alexander save as Rangers held on for a draw.