The Tigers sacked Ian Millward on Tuesday and Orr has been placed in temporary charge until a successor is found.
"I said I'd do the best I can for as long as I'm needed," he told Castleford's official website. "I'm here and I'll do the best I can.
"The guys have been brilliant. They're a good group and they've done everything that I've asked of them.
"The form's not been great so they're a bit down in the dumps but they feel like they've had a really good week and have got a lot out of it."
Castleford host Widnes Vikings on Saturday.