The home side started the game well as quarterback Teddy Bridgewater, who was making his first NFL start, found Jarius Wright for 52 yards to allow Matt Asiata to run into the endzone at the end of the drive.
However, the Vikings showed fight and Cordarrelle Patterson ran for a huge gain on the kickoff return for 49 yards to set up his side in good field position.
Bridgewater used that to good effect as he found Greg Jennings for 29 yards and Asiata scored his second touchdown of the game from six yards out.
Patrick DiMarco levelled the scores midway through the second quarter by finding enough space to collect a short pass from Ryan for the touchdown.
Minnesota were back ahead before the break when Bridgewater finished off a nine-play drive by rushing over from 13 yards, before extending their advantage with a Blair Walsh field goal.
In the second half, Walsh added another field goal before Atlanta mounted their comeback when Ryan delivered a 36-yard strike to Devin Hester, who beat a couple of tackles to run into the endzone for the score.
The Falcons took the lead when Antone Smith found space in the line and burst through to sprint his way down the right side of the field for a 48-yard touchdown.
However, Bridgewater rallied his side in the fourth quarter and produced a drive that saw them take the lead once more as Asiata got his third score of the game with a one-yard run.
The Vikings went for two and Bridgewater completed the conversion as he found Rhett Ellison in the endzone. Walsh then fired his side to victory with two late field goals for their second win of the season.