Following the London Olympics, both cyclists expressed a desire to be part of the team pursuit squad in the velodrome at Rio.
The British Cycling endurance coach told Sky Sports News: "We would like to break the world record. That is the benchmark and what we are aiming at. But it takes a huge commitment for an athlete to do that.
"They can't just walk in from the Tour de France and say, 'Oh, by the way, I want to be a team pursuit rider now'. It doesn't work like that any more.
"The event has moved on and for an athlete to commit that much time will impinge on their road career, so that is the athlete's choice. I think two years. It's a huge commitment. I don't know whether a road career will allow that."
Wiggins is one of Britain's most successful Olympians, having won four golds, a silver and two bronzes during the last four Games, while Cavendish is yet to make the podium at an Olympics.