Carroll joined the Hammers from Liverpool for a club-record £15m fee in the summer, but he has not yet featured this season because of injury.
The 24-year-old was initially out of action with a heel problem, but his comeback has been delayed by a double fracture in his right foot.
"Had we known he would be out for this long, we would not have signed him," Sullivan told BBC Football Focus.
"We are not a rich enough club to deal with that. You know any player can get injured, but we can't buy a player knowing he is going to be out for half the season.
"When we signed him we were assured by the medical staff that the very, very latest he would be back was September 1. That would have meant he would only miss two league games."
Carroll is expected to be fit to play in three weeks and Sullivan is hopeful that the England international can prove his worth when he returns.
He added: "If you ask me the same question at the end of the season, I hope I will be able give you a different answer and say 'Yes, it was the right decision to sign Andy Carroll'.
"He is a fantastic player and on his day he is unplayable. We love him and that is why we signed him. He is desperate to come back; he is sweating blood in training."
Carroll scored seven goals in 24 Premier League appearances during a loan spell at West Ham last season.