The visitors took the lead after less than four minutes in Sarajevo as Thomas Meunier broke the deadlock, but Bosnia were ahead by half time thanks to two goals in nine minutes.
Belgium also struck twice in nine minutes in the second half to swing the match back in their favour, though, and Dario Dumic's late equaliser for the hosts was immediately cancelled out by a winner from Yannick Carrasco.
Find out how an action-packed match unfolded courtesy of our minute-by-minute updates below.
This is the first match to be held in this stadium since before the war in Bosnia, and the hosts will be hopeful of marking the occasion with a crucial victory which would make their quest to finish in the runners-up spot a lot easier. Belgium have already done their part and will be at the World Cup next summer regardless, but they could still have a telling impact on who joins them there.
First, though, let's have a look at the team news for this one...
Well it is no surprise to see Bosnia go with near enough a full strength side for this crucial match, and that of course means Edin Dzeko leading the line. The former Manchester City man has continued his fine form at club level into the current season and comes into this match having scored seven times in just six appearances for Roma already this season. The striker also has five goals in the current qualifying campaign, so the hosts will be looking to him more than anyone to fire them to glory today.
Dzeko already leads the way when it comes to goals, scoring an impressive 52 times in his 87 international outings to confirm himself as the country's greatest ever goalscorer. Vedad Ibisevic also brings experience and goals to the squad with 28 strikes in 80 appearances, but he does not make the starting XI today.
Asmir Begovic of Bournemouth retains his place in goal for the hosts, while Arsenal's Sead Kolasinac continues on the left side of defence having made a solid start to life in English football. There is also a start for Everton midfielder Muhamed Besic, who comes into the side.
In addition to Besic, Dumic, Sunjic, Duljevic and Medunjanin all come into the side having missed out from the start against Gibraltar, while Cacalic, Zukanovic, Hadzic, Jajalo and Kenan all drop out of the starting lineup.
The Red Devils are likely to miss Lukaku, though. Aside from his incredible form at club level for United, Lukaku has also scored 10 goals in qualifying so far, which is only behind Cristiano Ronaldo and Robert Lewandowski in the European section. Overall he now has 24 goals for Belgium and is closing in on the all-time record of 30.
Michy Batshuayi will lead the line in his absence having impressed during the brief chances he has been afforded at Chelsea so far this season. Batshuayi could have more game time coming up due to Morata's injury, though, so he will be keen to get into some good form for Belgium and then try to take that back to Stamford Bridge.
Carrasco offers yet more threat in a wide area, and if all else fails then there aren't many better Plan Bs than Fellaini, who starts today having scored four goals in his last five appearances at club level.
Between them is the former Arsenal man Thomas Vermaelen, while Chelsea's Courtois is named between the sticks as expected. It is a hugely experienced Belgium squad, with 12 players boasting more than 50 caps - and that number could grow tonight should Defour come off the bench to reach his half-century.
As things stand Bosnia are in the driving seat in the race for second in Group H, but that could all change tonight. Three teams could still realistically finish second, with Greece just one point behind Bosnia and Cyprus three behind Greece. Victory for Bosnia in both of their final two games will send them into the playoffs, but this is a huge test for them tonight.
Defeat or even a draw for Bosnia in this match means that Greece would leapfrog them into second with a win over Cyprus, and considering Greece then host lowly Gibraltar in their last match of the campaign it would be a safe assumption that they wouldn't give second place back up from that position. Bosnia will be expected to beat Estonia in their final game, but it could be that they need something here to stay in control of the race.
Mehmed Bazdarevic's side had been unbeaten since their last meeting with Belgium before taking on Cyprus, but a surprise 3-2 defeat during the last international break means that they are now under pressure to pick up a result here. It is exactly a year ago to the day that Belgium ran out 4-0 winners in the reverse of this fixture, but Bosnia cannot afford a similar result here.
Bazdarevic's side are unbeaten in their last nine matches in front of their own fans - a run which includes nine wins and stretches back to another defeat at the hands of Cyprus - 2-1 in September 2014. It is more than three years since Bosnia were beaten on home soil.
Bosnia have kept clean sheets in each of their last four home games, last conceding a goal in front of their own fans in November 2015. That strike, in a 1-1 draw with the Republic of Ireland, is the only goal they have let in in their last seven home outings, and another clean sheet here would make it five on the bounce for the first time since November 2011.
Tonight's hosts have picked up 10 points from a possible 12 in front of their own fans, though, scoring 12 and conceding none, so Belgium will have to do what no other team has managed during this campaign and score away to Bosnia if they are to come away with all three points today.
On that occasion the Bosnians won eight and lost just one of their 10 matches to qualify quite comfortably, but there is a very real danger that they won't make it to the tournament at all this time around.
Roberto Martinez's side had their place in the tournament secured by the last international break, with a narrow 2-1 win in Greece sealing their ticket. It was a match which saw all three goals go in during a manic four-minute period, but Belgium ultimately got the job done to move unassailably clear of Bosnia.
It has almost been a perfect qualifying campaign for Belgium too, with just two points dropped from their eight outings so far.
Most impressive, though, has been their goalscoring. Lukaku and co have plundered 35 goals in their eight matches - a tally no other side could beat prior to the latest round of fixtures. They have been tight at the other end too, with just three goals conceded meaning they also have the joint-most miserly defence in qualifying.
That includes separate hauls of six goals, eight goals and nine goals, which is some improvement on the scoring front as before the current campaign Belgium scored three or more goals in a game on just one occasion from their previous 19 World Cup qualifiers.
Another win this evening would make it five on the bounce for the first time since October 2015, while also extending their unbeaten record to 12 matches across all competitions. Belgium have not lost since a 2-0 defeat to Spain last September, winning eight and drawing three since then.
In all qualifiers Belgium have only lost one of their last 28 outings stretching back to 2011, winning 22 of those and losing only to Wales in June 2015.
The last time Belgium failed to win a World Cup qualifier on the road was their 2-0 defeat in Estonia in 2009 - a result which at the time was Belgium's fourth away loss in a row and eighth without a win. It is safe to say that not many expected a run of nine consecutive wins to follow.
This is a tough one to call this evening. Belgium have named a strong side and look like the favourites on paper, but a positive result would mean so much more for Bosnia. Any dropped points for the hosts would open the door for Greece to leapfrog them later, but they are coming up against the best team in the group by some distance. I'm going for a score draw.
Sports Mole says: Bosnia 1-1 Belgium
Each country has won three and drawn one, although all of Bosnia's victories came in a row between 2005 and 2009. Tonight's hosts are winless in their three most recent meetings with Belgium.
A Spahic own goal broke the deadlock in the first half before Hazard, Alderweireld and Lukaku all got on the scoresheet too. It was a more convincing win than many expected, and Bosnia will remember that tonight.
The Red Devils have never beaten Bosnia in their own back yard, drawing one and losing two of their three previous visits. The most recent one of those came during qualifying for Euro 2016, when Nainggolan cancelled out Dzeko's opener in a 1-1 draw.
BOSNIA STARTING XI: Begović; Vranješ, Dumić, Šunjić, Kolašinac; Bešić, Lulić; Duljević, Medunjanin, Višća; Džeko
BELGIUM STARTING XI: Courtois; Alderweireld, Vermaelen, Vertonghen; Meunier, De Bruyne, Fellaini, Carrasco; Mertens, Hazard; Batshuayi
The full-back is involved in the build-up as he plays the ball out to Hazard before continuing his run into the box. Hazard returns the favour with a low cutback which finds Meunier, and he slams the ball home. It took a couple of deflections on the way through, but it looked to be going in anyway.
Dzeko is so often the goalscorer, but this time he turns creator by collecting the ball in the box and reaching the byline before squeezing a cross into the middle. His delivery crucially takes a deflection off Vertonghen and then a helpless Courtois, leaving Medunjanin with the simplest of finishes at the back post. He will never score an easier one!
The Tottenham defender completely misjudges a huge hoof forward from Begovic, and that allows Visca to suddenly find himself clean through on goal. The forward takes his time, but not enough for the Belgium defence to get back and he drills his low strike past Courtois.
Victory is the only way for the hosts to ensure that second place remains in their hands heading into the final game, and that is the case right now. Bosnia needed to come from behind, but they have responded really well to an early setback and take the 2-1 lead into the break.
This time it was a mistake from Vertonghen which led to the goal, with the defender misjudging a long ball forward to allow Visca clean through on goal. The shot itself was quite close to Courtois, but he couldn't keep it out.
Begovic did almost score an own goal shortly afterwards, but he was saved by the woodwork having tried to intercept a low cross from Hazard.
Carrasco is the creator with a brilliant turn inside the area to escape the attentions of his marker, but his low strike is straight at Begovic. The keeper cannot hold on to it, though, spilling it into a dangerous area which allows Batshuayi to tuck it home from close range. Begovic will be disappointed with that one!
A corner is flicked on by Meunier towards the back post where the Tottenham man is waiting, and he is left with a simple finish. Medunjanin was on the line, but he didn't fancy coming out to challenge Vertonghen by the looks of things!
The resulting corner is played into a good area, and Dumic is there to plant his header home from close range. It is dreadful marking from Belgium, but Bosnia do not care one bit.
Meunier lifts a clever pass into the path of Carrasco, who tucks his finish past Begovic at the near post. What a game this is!
When all is said and done, though, it is a hugely damaging defeat for Bosnia. Things are now out of their hands in the race for second place, and should Greece beat Cyprus later tonight then they will have all but secured second place with only Gibraltar to come after that.
There are positives for Bosnia from this performance, which will go down as their best for some time, but ultimately they came up short this evening.
Thank you very much for joining Sports Mole for this thrilling contest as Belgium dent Bosnia's World Cup hopes with a topsy-turvy 4-3 win in Sarajevo. I will leave you with our match report, and be sure to stick around for coverage of the 7.45pm kickoffs too. From me, though, it is goodbye for now!