Vogel, 42, was out of contract at the end of this season, which ended for the Pacers recently when they lost game seven of their Eastern Conference first-round playoff match against the Toronto Raptors.
"I've decided that it's time for a new voice around here," Bird told reporters. "This is one of the toughest things I've done.
"Frank's not getting fired, his contract is up and I just made the decision not to renew it."
Vogel leaves after five-and-a-half seasons with the Pacers, where he managed a 250-181 regular-season record and made the playoffs five out of six seasons.