Froch controversially stopped George Groves in the ninth round of their world title contest on Saturday evening, and while Ward has insisted that he would be prepared to face the Nottingham fighter for a second time, he has revealed that he wouldn't be overly keen on competing in England.
The 29-year-old told BBC Radio 5 live: "From a legacy standpoint, Froch needs me more than I need him. I can go off to fight other guys and never mention Carl Froch's name again.
"I'm not concerned about fighting in front of his fans, but it is about not getting a fair shake. I've realistic concerns.
"Like I said, it's not my legacy at stake if he didn't beat me, but if something happens like in the Groves fight, then that is tough to live down. It's a lot of moving pieces."
Ward outclassed Froch during their first encounter in Atlantic City in 2011.