The African side looked on course to pull off an upset in Group H when Sofiane Feghouli put them ahead from the spot after 24 minutes.
However, Belgium hit back with two goals in the final 20 minutes to turn the game on its head, leaving Halilhodzic wondering what could have been.
"It's a major disappointment," the 61-year-old is quoted as saying by BBC Sport. "I would have liked to have a couple of tougher and more resilient players to resist their attacks but I don't have much of a choice.
"We missed a great opportunity here. But in the second half it was a bit tough. We left too much space to the Belgian team and the two goals scored against us were to be expected."
Algeria return to action on Sunday when they take on South Korea.