The running back has been disappointing during Indiana's warm-up fixtures, contributing just 2.4 yards per game so far, which measures up as less than his disastrous 2.9 average last campaign.
"Trent, he needs to answer the bell and do his job to the best of his ability. We're all accountable here," said Grigson, according to ESPN.
"I will say this, there are a lot of backs last year that wouldn't have got [2.9] considering the amount of people he had in that box and the amount of bodies that were hitting him before he even seemed to get the ball sometimes.
"He's such a hard runner, we know how tough he is, but he's got to produce just like all these guys do on this final 53."
Richardson will get the chance to prove his real worth when the new NFL season gets started next month.