The 45-year-old has overseen just one victory in 13 Championship matches this season, and the last six have ended in defeat.
As a result of Rotherham sitting at the bottom of the league table Stubbs has lost his job at the New York Stadium.
A statement on Rotherham's official website read: "Following a board meeting this morning it was felt that a change of first team management was needed to give the club the best possible opportunity to preserve our Championship status.
"The club would like to place on record our thanks to Alan, John and Andy for their effort and professionalism, and we wish them well for the future.
"The board have now started the process of appointing a new first team manager. The club will be making no further comment at this stage."
The Millers' next game will be against Reading at home on Saturday.