The Scotland international was deemed surplus to requirements at Portman Road in 2009 by Roy Keane, who managed Ipswich at the time.
However, Rhodes has gone on to score over 100 goals for Huddersfield Town and Blackburn Rovers since his departure from the Suffolk-based club.
With that statistic in mind, Clegg has admitted that Ipswich should have held on to the 23-year-old for longer.
"Hindsight is a wonderful thing and with the benefit of that, yes, we would have kept him," Clegg told the East Anglian Daily Times. "Jordan has done incredibly well in League One and the Championship since leaving Ipswich and I hope he looks back on his time at Ipswich with fond memories.
"But let's be clear, ever since myself and Marcus Evans have arrived at the club, we have trusted in the judgement in the manager."
Clegg brought an end to his four-year stint at Ipswich yesterday.