© PA Photos
The Premier League side had the better start of the two teams, but there were few chances in the opening 10 minutes.
The visitors took the lead after 22 minutes, when Darren Bent scuffed the ball into the back of the net off the back of a Villa counter-attack.
The advantage was short-lived, however, as Danny Shittu equalised for the home side just five minutes later, managing to lose his marker during a corner to power the ball home.
Villa looked edgy every time their Championship opponents put the ball in the box, Shay Given often looking to take the ball early.
Play was stopped for a time after the break while debris was cleared from the pitch, after fans threw several bottles onto the field.
It looked as if a replay was on the cards, but Millwall scrambled forward in the 89th minute, John Marquis eventually heading the ball in to put the hosts ahead.
© PA Photos
The result sees Paul Lambert's side exit a cup competition at the hands of lower league opposition for the second time in four days, having been knocked out of the Capital One Cup by League Two Bradford City on Tuesday.