Former Cardiff promotion-winning captain Mark Hudson has taken caretaker charge of the Terriers and they looked like they would ease their problems with a battling point.
Siewert was dismissed following Huddersfield's defeat to Fulham last Friday.
Joe Ralls put Cardiff into the lead in the 42nd minute before Trevoh Chalobah levelled early in the second half. It looked like it would be enough for a point, but Cardiff substitute Hoilett won it two minutes from time.
Cardiff gave a first league start since December 2017 to Lee Tomlin as one of four changes from their 3-0 hammering at Reading on Sunday.
Huddersfield made three of their own, but it was their opponents who started quickly as winger Josh Murphy had a curled effort tipped over by Kamil Grabara.
Gavin Whyte then had a shot deflected wide before Robert Glatzel missed a sitter.
Tomlin's ball found Cardiff's £5.5million summer arrival Glatzel, who looked like he thought he was offside, but the flag stayed down and he somehow fired over with the goal gaping.
At the other end, Rajiv Van La Parra found the side netting after being played in by Pritchard.
As Reading had done, Huddersfield bypassed Cardiff's press with ease and, after Karlan Grant made a fool of Sean Morrison, Jonathan Hogg forced Alex Smithies into a fine save.
But Cardiff built up a head of steam before the break and finally broke the deadlock when Aden Flint's knockdown fell to Ralls who hit the net with his weaker right foot in the 42nd minute.
Glatzel and Joe Bennett came close to doubling the Cardiff lead before the half-time whistle.
Huddersfield levelled the game up at the start of the second half as Cardiff allowed Chalobah too much space on the edge of the box. It proved costly as the former Ipswich man found the bottom corner with Smithies helpless.
Chalobah had been one of the best players on the pitch until that point and after his strike Huddersfield held firm in the wake of a physical Cardiff assault.
Glatzel continued his one-man mission to grab a goal in the final quarter, but couldn't come up with the goods despite being presented with a series of chances.
Murphy was wiped out in the air by Tommy Elphick on the edge of the Cardiff box and substitute Omar Bogle hit the crossbar.
It looked as though it would sum up a night of frustration for Neil Warnock's men, but Hoilett latched onto Murphy's pass and struck a deflected winner in the 88th minute.