It was Kamara who went closest in the opening period, heading just wide following a flurry of chances for Crew.
Gaston Sauro then deflected a Giovinco effort wide of the target.
In a tight affair, Finlay broke the deadlock after 40 minutes as he fired past Toronto keeper Joe Bendik for his 10th of the season.
A dipping Giovinco free kick in the 62nd minute was turned around the post by Columbus stopper Steve Clark.
Higuain's 76th-minute right-footed effort smashed the crossbar as Crew looked to increase their lead.
The Argentine struck the upright again 10 minutes later as Columbus continued to threaten.
It was third time lucky for Higuain, as he chipped home in injury time from Aaron Schoenfeld's flicked header to hand his team all three points.
The 1-0 victory moves Crew up to second in the Eastern Conference table, ahead of Toronto who drop to third.