The fixture was the first ever Bundesliga derby between the two Berlin-based sides, with Union promoted from the second tier last season.
The clubs previously sat on opposing sides of the Berlin Wall, which fell almost exactly 30 years ago and divided the east and west of the city.
Union's Christian Gentner threatened to open the scoring in the third minute, connecting with a Sebastian Andersson cross but seeing his effort rebound off the post.
Union then managed to break through Hertha's defence again as Gentner took a right-footed shot which was easily kept at bay by Jarstein.
The game remained goalless at the interval, with play paused in the early stages of the second half due to fireworks let off by fans.
A spell of defensive play followed, with only Union creating a chance when substitute Joshua Mees attempted to hand his side the advantage.
His strike was blocked by Hertha's Dedryck Boyata before a follow-up effort from Andrich curled wide.
The deadlock was finally broken in the 90th minute, when Mees was brought down and a penalty was awarded to Union.
Polter stepped up to the mark and neatly converted the opportunity.
Ten minutes of added time followed due to the earlier delay, but Hertha could could not respond and found themselves defeated by the team on their doorstep.