The Ireland international broke his arm during the 23-21 victory in the opening Test against Australia on Saturday, and has now been ruled out of the remainder of the series.
While O'Connell could still stay with the group in a supporting role, Rowntree is gutted to lose someone in the form that the 33-year-old was in.
"He is one of the best, and he's in the form of his life as well," Rowntree said. "He came into selection quite late because he had been out for so long with injuries during the season. Suddenly, he came back on the radar after that game [for Munster] against Harlequins in the European Cup. He was fit. In fact, I don't think he's ever been fitter.
"But you have got to get on with these things, which is why we brought so many players. Whether he is going to stay I am not sure yet. I would like him to because of the influence he has around the group. We will find out in the next four or five days what he is going to do.
"Losing someone like Paul galvanises a squad like us. We are a very tight group."
The Lions face Melbourne Rebels at 10.40am on Tuesday.