In February, Grayson arrived at Valley Parade to try to boost the club's hopes of ending the season in the League One playoffs, but just 14 points were recorded from the 48-year-old's 14 matches in charge.
After talks were held between Grayson and the club's hierarchy, the former Bantams defender has opted to walk away from his role in the dugout.
In a statement, Grayson said: "After conversations with the owners I have decided to turn down the option to extend my contract. This was an extremely difficult decision as I have enjoyed my time at Valley Parade.
"I would like to thank everyone at Bradford City, from the owners to the supporters, who have turned out in their numbers both home and away since I have been manager of their club. I wish the club all the best for the future."
Bradford ended the season in 11th place in the standings, eight points away from the playoffs.