The Scottish Premier League outfit have stumbled to four defeats in a row and currently sit second from bottom in the table.
A spokesperson told the club's official website: "When John came to the club, Hearts was embarking on a transitional phase in the club's history. He began that important period of change and it is now imperative we find the right man to take the transition forward.
"The over-riding factor in this decision was the long-term well-being of Heart of Midlothian Football Club. We would like to thank John for his tireless efforts as manager of Hearts and wish him well with his career. He will be welcomed back to Tynecastle as guest of the club at any time."
McGlynn told Hearts News: "I was very grateful to be given the chance to become the Hearts manager as it was my dream job. It was an honour and privilege to serve the club in this capacity and I thoroughly enjoyed working with the players, who have been top professionals.
"It is unfortunate that things didn't turn out the way I wanted them to and I am gutted about that. I am a worker and want to get back into football as quickly as possible."