The 36-year-old has been playing for the Scottish club since 2009, making more than 200 appearances, following his move from Livingston.
Since the club's Scottish Cup triumph in 2014, Mackay has struggled with a hip injury, which required two operations, but the defender feels as though he is no longer physically strong enough to cope.
"I've been aware of this issue for the past couple of years and although the first operation brought about a level of improvement that enabled me to get back playing and feeling fine, there was a gradual deterioration again over a period of time to the extent that I was unable to train," Mackay told the club's official website.
"A second operation followed which again helped but not to the extent that I feel that I can get back to the fitness levels needed for full-time football. The decision hasn't been an easy one but I've had to take into consideration that I don't want to do irreversible damage that affects me away from football - I'm only 35 and have a lot of years ahead of me that I don't want to be affected.
"I've probably known in my own mind that this decision had to be made and I'm happy that I've given myself every chance to get back playing but it's not to be."
Mackay, who previously played for Dundee and Oxford United, will stay involved with St Johnstone as he continues to help coach the Under-20s.