Prior is struggling to make a full recovery from an Achilles injury, and Cook believes that they simply can't afford to rush the wicketkeeper back for the first Test on June 12.
"It is a big concern," the England skipper conceded when asked about Prior's fitness.
"I've been speaking to him every week, and he's been doing everything he can. He's incredibly frustrated, because Achilles injuries are hard to heal.
"He's a big part of our plans, but we can't rush a guy back. That wouldn't be fair on him or to the England team."
Prior was dropped from the side after three matches in the Ashes series during the winter.