The Terriers have not won a game in the Championship since November 10 and they now sit in 18th place in the league table.
Chairman Dean Hoyle told the club's official website: "In making this decision, we have thought long and hard. It has been made with both the short term and long term needs of the club in mind. We brought Simon in with the immediate aim to lead the club to promotion and he did that at the first attempt. It is an achievement for which we are grateful.
"We knew the transition to Championship football would be difficult and our targets for this season have been realistic, with retention of our status the priority.
"However, having worked so hard to get back into the second tier we have to do everything we can to consolidate that position. After exceeding all our expectations in the early part of the season, our results and performances as a team and from individual players have tailed off alarmingly."
Grayson won 17 of his 49 matches in charge of Huddersfield.