It looked almost certain that these two well-matched teams would cancel each other out after the advantage Harvey Elliott's sumptuous curling effort provided for Rovers in the 25th minute was wiped out by Scott Malone's thunderbolt nine minutes later.
The Lions thought they had done enough for what would have been a hard-fought point, but Armstrong popped up to take his Championship tally for the season to 14 after his shot found the net in the first minute of stoppage time.
Rovers have secured three wins on the spin for the first time since November 2019 and remain within striking distance of the top six, while Millwall's five-match unbeaten run came to an end, although they have gone seven without victory.
In-form Armstrong caused havoc with a piercing run early on before the ball fell to Elliott, who set himself up for a fine low volley that goalkeeper Bartosz Bialkowski did well to parry.
The game was tight in the opening quarter but something special broke the deadlock after 25 minutes as Armstrong's perfectly-weighted ball found Elliott on the right and the 17-year-old opened up his body before expertly curling a shot that flew past the Millwall keeper and into the far top corner.
Millwall rallied and Jed Wallace saw an effort deflect agonisingly wide. He was involved 11 minutes from the break as the Lions deservedly equalised. His surging run found Malone on the left and he hammered a side-foot shot into the top corner.
Elliott went close before the break but could not get a touch on Armstrong's inviting ball and shortly afterwards Jon Dadi Bodvarsson got the better of Ayala but his turn and shot was repelled by the legs of Thomas Kaminski.
Chances fell the way of Rovers as the game wore on. Substitute Tyrhys Dolan ought to have done better when he found himself beyond the Millwall defence but lobbed tamely at Bialkowski, and Armstrong fashioned space for a shot moments later but it was charged down.
Just as it looked like the spoils would be shared, Rovers broke Millwall hearts in the first minute of time added on.
Stewart Downing was the architect, weaving to engineer space, and when his shot was blocked, the league's leading marksman Armstrong reacted quickest, smashing a 12-yard shot beyond Bialkowski via a deflection.
There was still time for Tom Bradshaw to see a shot blocked and in the dying seconds Millwall were convinced a Rovers handball should have given them a penalty but the referee disagreed.