The 35-year-old took the reins at the Proact Stadium in January and oversaw their relegation from League One last term.
A disastrous start to the new campaign has seen the Spireites register just one win and five defeats from their opening eight games, leaving them 23rd in the table, and now the club have decided to act.
Ashley Carson, director at Chesterfield, said in a statement: "Gary has worked very hard during his time at the club, but we find ourselves in a lowly position after providing him with substantial backing in the transfer market.
"After making a very poor start to the season, we feel that now is the time to appoint a new manager to bring about a change in fortunes."
Caldwell leaves with a record of just three victories from his 29 games in charge - a win percentage of 10.3%.