The decision comes following the German club's 2-1 defeat to Hamburg at the Imtech Arena on Friday.
The result left Bayer fourth in the Bundesliga with 48 points from 29 matches, just a point clear of fifth-placed Wolfsburg, who have played a game fewer.
The 2002 Champions League finalists have suffered a dip in form since the turn of the year, having been able to collect just 11 points in 2014.
Bayer were also knocked out of the DFB-Pokal by second division club Kaiserslautern and were comprehensively defeated by Paris Saint-Germain in the Champions League last 16.
"It was a very difficult decision as Sami has done a fantastic job for us in the last two years," said the club's sporting director Rudi Voller.
"What he and his coaching staff managed to get out of our team in the first half of the season far exceeded our expectations and earned him widespread praise. It makes this decision all the more difficult.
"However, the developments of the last few weeks gave us no other option but to go our separate ways. We have to do everything in our power to save our season."
Bayer's youth team head coach Sascha Lewandowski has been appointed as the club's interim first-team manager.
The 42-year-old worked alongside Hyypia as the Bundesliga club's joint-manager two years ago, before the former Liverpool defender took sole charge of the team last summer.