The IBF announced just hours before the hearing was scheduled to start in New Jersey that the Englishman's team had changed their minds without giving a reason.
"The IBF received notice that Golden Boy Promotions withdrew their appeal of the Khan-Peterson bout," read a statement from IBF public relations director Jeanette Salazar.
"The hearing that was scheduled for tomorrow (Wednesday) has been cancelled. We have not received any further information on this decision by Golden Boy Promotions."
The Bolton fighter is still expected to attempt to reclaim his IBF and WBA belts in a rematch against the American.