Substitute Bacuna pounced in the 82nd minute to fire home for his first goal since March.
And his effort proved enough to secure Huddersfield their first away league win since November 2018.
However, defeat for Stoke sees Jones' failing side rooted to the basement of the Championship and put his position as manager under more strain.
Jones stressed in his pre-match press conference that he did not fear the sack.
But conscious of avoiding yet another defeat the Welshman made three changes to his starting side.
Former England goalkeeper Jack Butland was dropped after what has been a calamitous start to the season.
His latest error saw Stoke beaten 3-2 at home to Nottingham Forest last Friday night, so in came replacement Adam Federici.
Huddersfield boss Danny Cowley resisted the temptation to change his side despite having not won in his first three games in charge.
A credible 1-1 home draw with Millwall at the weekend meant Cowley kept faith with his team, including Liverpool loan keeper Kamil Grabara.
Grabara's handling error denied the Terriers their first home win of the season, but the Polish keeper was given another chance to impress.
The visitors should have been ahead in the fifth minute but Elias Kachunga badly fluffed his lines. Karlan Grant teed the winger on the edge of the six-yard box but, with only Federici to beat, he miscued horribly.
Midfielder Tom Ince lashed a left-foot shot well wide moments later but the next chance was not until just before half-time as both teams cancelled each other out.
Cameron Carter-Vickers whipped in a lovely delivery from the right touchline but Allen powered his header over the bar.
Danny Simpson's interception fell fortuitously into the path of Carter-Vickers three minutes after the restart, but his vicious dipping drive flew inches over.
Isaac Mbenza was also brought on by Cowley as another attacking option – and how this game needed it.
Gregory headed well over from a Sam Clucas cross with 20 minutes remaining, but the home side could not find the back of the net.
Substitute Campbell rifled wide of the left post as the Potters countered, but again there was a lack of quality with the finishing.
Gregory poked wide at the near post but it was Bacuna who won it at the death with his first goal of the season.
It was Cowley's first win as manager and a massive boost for the Terriers who have waited nearly a year for an away win, their first of the season.