The 47-year-old took over at the Stadium of Light in June following the Black Cats' relegation from the Premier League but has overseen a disastrous start to their first season back in the Championship, registering just one win in their first 15 games.
News of Grayson's departure - alongside that of his assistant Glynn Snodin - came just 16 minutes after the final whistle in the side's 3-3 draw with bottom side Bolton Wanderers tonight, a result that extended their long run without a win at home to 318 days.
Sunderland chief exec Martin Bain said: "Simon and his team have worked tirelessly to achieve the best for the football club during their time here.
"While we hoped that Simon's experience in the Football League would help us to a successful season, results have not been good enough for a club of this stature.
"In order for us to improve upon our current position we believe a fundamental change is necessary."
Grayson, who had earned plaudits for his work with Preston North End prior to his move to Sunderland, departs with the club 22nd in the Championship table and two points from safety.