Perisic has been heavily linked with a transfer to Old Trafford throughout this summer, with Jose Mourinho understood to have made the 28-year-old one of his primary transfer targets.
Inter are holding out for a fee of almost £50m, but club captain Icardi believes that Perisic should leave if he does not want to be at San Siro.
"It's not something we talk about in the dressing room," he told Italian broadcaster Premium Sport.
"If he's not happy then he has to leave, as the coach (Luciano Spalletti) says. He needs to do what makes him happy."
Perisic scored 11 goals and created 12 more during his 42 appearances across all competitions last season.