Rivers has since moved on to the Los Angeles Clippers, but returned to Boston this week for a charity event where he was asked whether he thought it was a mistake to let Perkins leave.
"I think so, because we needed the toughness," Rivers said according to NESN. "The one thing we did by losing Perk was, we removed Kevin's protector.
"I didn't think it was a coincidence that, after Perk left, that Kevin got into all those little flicks with the other teams. Perk deflected all that."
While Jeff Green is set to continue to be an important part of the Celtics' roster next season, Rivers believes that it may have been wise to keep hold of his former star last year.