The defender has recently been linked with a move away from Anfield following a lack of first-team action last season.
Olsen has now suggested that Agger may need to leave the Reds in order to improve his form at international level.
"You may ask Daniel about [it]," he told Bold. "He can best gauge how it looks now and, more important, next season.
"But of course he should be playing. Otherwise, I believe we will have a frustrated and irritated captain. I hope he will make the right choice."
Agger joined Liverpool from Brondby for £6m eight years ago.