The 31-year-old, who also opened the scoring, struck with nine minutes of the additional 30 remaining at the Mapfre Stadium as Gregg Berhalter's side advanced with a dramatic 4-3 aggregate victory.
Kamara continued to run amok and then headed Harrison Afful's delivery against the crossbar as Columbus looked to improve on their fast-and-furious start.
Montreal were reduced to long-range efforts in a half which saw them fail to get behind the hosts' defence, with Ignacio Piatti firing narrowly over the bar on the half-hour mark.
However, Dilly Duka secured parity at the break by turning the ball home from close range on 40 minutes after the Crew failed to clear their lines from a corner.
It was Finlay who spared his teammate's blushes by smashing home into the roof of the net with 12 minutes to play as Berhalter's side forced extra time.
A cagey first half passed without incident, before Kamara headed home Cedric's flick-on at the near post to seal their progression into the finals.