The Danish defender was sold to Brondby during the summer after spending eight years at Anfield, with reports suggesting that he could no longer handle the demands of Premier League football.
However, Agger has revealed that he left the Merseyside club because he could not get on with Rodgers.
"Let me put it this way, the manager and me were perhaps not on the same page in all of last season. For most of last season at least," the Daily Mirror quotes Agger as saying.
"There was much between us, and for me it was just enough. I felt that he didn't appreciate the things I could and contributed. When I feel that, then it's time to move on.
"Where me and him maybe went wrong from each other, was that I'm very direct. I say things as they are, and so I also expect that people are the same to me. Maybe it's wrong to expect it sometimes."
Agger scored nine goals in 156 appearances for Liverpool.