The 32-year-old striker scored 10 goals for the Gills during the 2013-14 campaign.
"Sadly we have been unable to agree terms on a new deal and so Bayo will be leaving the club," Taylor told Gillingham's official website.
"I have the utmost respect for Bayo; he has been a superb ambassador for Gillingham and scored some very important goals for us last season.
"He has been a model professional, a great character to have around the place and I have very much enjoyed working with him."
Akinfenwa joined Gillingham for the second time in July 2013 following a brief spell at Northampton Town.