The British middleweight, who won Olympic bronze in London last summer, was set to learn his opponent in the near future for the meeting in Las Vegas.
However, the 24-year-old has suffered an injury to his hand and he has now pulled out of the event on September 14.
"I've made the hard decision to pull out of this one and focus on getting fit," he told BBC Radio Suffolk.
"I could have had injections in it but it's only my fourth fight and I don't want to be going down that road too soon.
"It's a hard decision but I think the right decision."
Ogogo has won all three of his professional bouts.