Zeman, who was appointed as coach of the Rome side in June 2012, struggled in his time at the Giallorossi.
"We decided to sack Zeman because we thought it was the only decision to take," said Baldini. "He's not the only one responsible, but for now we can only change the manager.
"We have faith in Andreazzoli, he knows the players well as he's been here for years, which means we don't lose more time with him getting to know the players.
"After talking it over with the owner and the management we thought it was the right decision."
Roma currently sit in eighth place in Serie A.