The victory has seen Iceland finish second in the group behind Hungary, while Portugal will advance as one of the four best third-placed teams. Austria, however, have ended the campaign bottom of the section.
Iceland came within inches of taking the lead inside two minutes when Johann Gudmundsson struck a rasping effort from distance, only to see the ball smash off the right-hand post.
Iceland continued to threaten the Austrian goal, however, and made the breakthrough in the 18th minute when Jon Dadi Bodvarsson clipped the ball past Robert Almer from close range after Kari Arnason had met a cross from the right.
Austria had the chance to respond in the 23rd minute when Julian Baumgartlinger found space inside the Iceland box, but the midfielder's effort was always moving wide of the far post.
Marko Arnautovic then twice came close for Austria either side of the 30-minute mark, but the Stoke City attacker was just denied by Hannes Halldorsson after the goalkeeper took too long on the ball, before finding the palms of Halldorsson from distance.
A eventful first period then saw a penalty awarded as Iceland full-back Ari Freyr Skulason was adjudged to have pulled David Alaba inside the Iceland box. It was Aleksandar Dragovic that stepped forward for the Austrians, but his spot kick hit the outside of the post.
Iceland just could not keep the ball as the first period approached its climax, and Baumgartlinger again came close with a strike from distance before Martin Hinteregger missed the post in the 43rd minute.
Austria continued to threaten early in the second period and Arnason had to be at his best to block a strike from Alaba, before Christian Fuchs had an effort of his own blocked in the 48th minute.
Despite the pressure from Austria, Iceland might have had a second in the 49th minute when Gylfi Sigurdsson broke down the other end before driving the ball towards the bottom corner, but his effort was just wide of the post.
Austria were back on the front foot, however, and Alessandro Schopf twice came close to an equaliser 10 minutes into the second period. Eventually the equaliser did arrive on the hour when Alaba found Schopf and the midfielder tricked his way through the defence before finding the back of the net.
Sigurdsson again came close for Iceland in the 65th minute, but the Swansea City midfielder could not beat Almer from inside the box, before Marc Janko missed the target with a first-time effort down the other end.
It was all Austria as the final 15 minutes approached and Janko again missed the target in the 73rd minute as Iceland continued to ride their luck at the Stade de France.
The pressure continued into stoppage time, but Austria were caught on the counter in the fourth minute of added time when Birkir Bjarnason raced away and Arnor Traustason was on hand to convert at the far post and secure second spot for his team.
Iceland will take on England in their last-16 clash on Monday night.