Graham has had an excellent start to the season and has scored eight touchdowns from 40 receptions for 630 yards.
Cromartie was expected to cover the Saints' number one weapon himself but has admitted that task will fall to his side's safeties.
"No, I don't think so," Cromartie told the New York Daily News, when asked if he would be covering Graham.
"[From] Antonio Allen to Dawan Landry, I think we have some good cover safeties, so I don't see it happening."
The Jets are 4-4 in their season.