The Danish defender started the first three Premier League games of the season, but Reds boss Brendan Rodgers has since preferred to name Kolo Toure, Martin Skrtel and £17m-signing Mamadou Sakho in a back-three formation.
Agger has previously stated his commitment to Liverpool amid reported interest from Barcelona, but has now admitted that he will be forced to leave Anfield if he does not return to the starting XI soon.
"If I don't play, if the manager doesn't think I'm good enough, that could make me leave Liverpool," he told reporters.
"I'm here to play football, nothing else. So if I don't play football then I don't want to be here any more."
The 28-year-old, who has not started a Liverpool match since their 3-1 victory over Crystal Palace on October 5, signed a contract extension with the Merseyside outfit last year.