Van Commenee announced that he is to step down as head of the organisation yesterday.
Rutherford told Sky Sports News: "I am a bit upset with that to be honest. I really thought that Charles, if he had stayed on for another four years, could have put in some great changes which really could have benefited everybody at UK Athletics and the athletes.
"I was told by him that he didn't think I was good enough at times a few years ago to now having the full support of him. It is going to be a real shame to see him go."
Van Commenee leaves his post in December.