The hosts go into the final day needing 346 runs to level the series and remain top of the world rankings.
"As a team collectively over the last number of years, we've enjoyed breaking records and defying people's beliefs against us," Finn told Sky Sports News.
"So tomorrow, when we get here, we've got a 15-over old ball to bat against - which isn't going to do as much as an new ball obviously.
"We've got a great opportunity to assert ourselves on the South Africans early - and we really do believe in the dressing room that we can win this game tomorrow."
Finn has taken eight wickets and scored 10 runs so far in the series.