The 47-year-old's departure was announced just 16 minutes after the final whistle of the Black Cats' 3-3 draw at home to bottom side Bolton Wanderers, a result that extended their record for the season to just one win in 15 Championship games.
Grayson took over at the Stadium of Light in June following the club's relegation from the Premier League, after several other candidates had allegedly turned down the role due to concerns over a lack of transfer funds.
"I would like to take this opportunity to express my thanks to the club and all the fans, players and staff at the club's training ground and the Stadium of Light," Grayson said in a statement issued through the League Managers' Association.
"Sunderland AFC is a huge football club and whilst I am very disappointed not to be continuing as its manager, I hope that the club will stabilise and return the success and first-class football that the supporters and people of Sunderland deserve.
"I would like to wish everyone associated with Sunderland AFC all the very best for the future."
Grayson left Sunderland with the club 22nd in the Championship table, their lowest position in the league pyramid in more than 11 years.