The BayArena side have underachieved this season and sit ninth in the Bundesliga - 13 points away from a Champions League spot - after 23 games.
Saturday's heavy loss - their third consecutive defeat in the league - proving the final straw for the club's board.
"Bayer 04 have parted ways with head coach Roger Schmidt with immediate effect," read a statement on the club's official Twitter account.
The 49-year-old took charge of Leverkusen in 2014 after impressing during his time in charge of Red Bull Salzburg, and led them to fourth place in the table in his first season at the club, before going one place better in 2015-16.
Leverkusen will be looking to get their season back on track without Schmidt on Friday when they host Werder Bremen, before striving to overturn a 4-2 Champions League first-leg defeat against Atletico Madrid at the Vicente Calderon.