Madrid entered the match having already qualified for the round of 16 as Group H runners-up, but Dortmund needed a favour from section winners Tottenham Hotspur in a bid to secure third position ahead of APOEL.
As it transpired, APOEL suffered a 3-0 defeat at Wembley, meaning that Dortmund were able to secure the Europa League spot despite ending their Group H campaign with a narrow loss.
Real Madrid boss Zinedine Zidane made five changes from the team that started against Athletic Bilbao at the weekend.
It was Madrid that had the first half-chance of the match inside three minutes, but Dortmund goalkeeper Roman Burki just about did enough to keep out a powerful strike from Ronaldo.
Los Blancos were ahead in the eighth minute though, with Mayoral clipping home from close range after Isco's touch from a Ronaldo cross had found the 20-year-old in a dangerous position.
The hosts then doubled their lead in the 12th minute through a wonderful Ronaldo effort as the Portuguese became the first player in history to score in all six Champions League matches in the group stage of the competition.
Theo Hernandez then sent one just wide of the Dortmund post in the 22nd minute as Real Madrid threatened to run away with the contest before the mid-point of the first period.
Dortmund's best work was coming out wide as Raphael Guerreiro was given freedom to break forward, and the number 13 came within inches of finding Aubameyang with a low cross in the 25th minute, before Kagawa saw a close-range strike blocked by Raphael Varane after smart work from Pulisic.
Ronaldo again tested Burki in the 33rd minute as Real Madrid threatened a third, but Dortmund were seriously threatening down the other end, with Aubameyang sending the ball narrowly wide of the post from a dangerous position.
A Dortmund goal was coming, and it finally arrived in the 43rd minute when Aubameyang headed a super Marcel Schmelzer cross into the back of the net to change the complexion of the match before the interval.
Aubameyang then levelled the scores in the 48th minute when he chipped over Keylor Navas at the second time of asking after first being denied by the Costa Rica international. Real Madrid wanted offside, but the goal was allowed.
Ronaldo should have netted his second three minutes later after Isco fed the number seven inside the Dortmund box, but his low effort missed the far post.
It was like a basketball match in the period either side of the 60-minute mark, with both teams committing numbers on the counter-attack in a bid to snatch the game's fifth goal.
Ronaldo had a stunning chance to net his second in the 74th minute after Asensio delivered a brilliant cross into the Dortmund box, but the Portuguese somehow fired over under pressure from Neven Subotic.
Ronaldo had another opportunity three minutes later, but this time headed into the arms of Burki from close range. The former Manchester United forward then had a legitimate goal chalked off for offside as Dortmund came under increasing pressure.
Dortmund finally conceded a third in the 81st minute when Vazquez's clever effort found the bottom corner, and that proved to be the winner after Kagawa fired the away side's best chance over the crossbar late on.
Real Madrid (4-3-3): Navas; Nacho, Varane (Asensio 38'), Ramos, Theo; Kovacic (Ceballos 58'), Casemiro, Isco (Llorente 69'); Vazquez, Mayoral, Ronaldo
Dortmund (4-2-3-1): Burki; Subotic, Bartra (Toprak 80'), Sokratis, Schmelzer; Sahin (Weigl 65'), Dahoud; Pulisic, Kagawa (Yarmolenko 92'), Guerreiro; Aubameyang