Robbie Neilson's side laboured at Bayview, after the game was moved because of flood damage at Central Park.
They stepped up the pressure in the final quarter of the game and Halkett powered home Elliott Frear's 78th-minute corner for a 1-0 win.
Jamie Walker had a stoppage-time penalty stopped by Ross Sinclair but Hearts had already secured their second triumph.
Elsewhere in Group A, Raith Rovers came from behind to snatch a late 2-1 derby win against East Fife at Stark's Park.
Teenager Liam Newton opened the scoring 15 minutes into his professional career with a shot on the turn, but the hosts levelled a minute before the break when Regan Hendry netted from the spot.
Manny Duku grabbed the winner three minutes from time with a close-range finish.
Dundee won 2-0 at Brora Rangers in their first game of the season, following a false start in midweek when Covid-19 issues saw Forfar pull out of a trip to Dens Park.
With Charlie Adam making his Dundee debut, Graham Dorrans fired home in the 11th minute and Danny Mullen netted at the near post 27 minutes in.
Arbroath recovered from an early blow in their Angus derby with Montrose to make it two wins from two with a 3-1 result.
Graham Webster opened the scoring for the visitors at Gayfield inside 10 minutes but Gavin Swankie and Jason Thomson turned the game on its head before half-time.
Thomas O'Brien netted a brilliant lob with four minutes left to ensure Arbroath went four points ahead of Ross County at the top of Group D.
Elgin City are in second place after goals from Kane Hester – who was later sent off – and Russell Dingwall earned a 2-0 home win over Stirling.
Clyde edged a five-goal thriller against Dumbarton at Broadwood to get their campaign off to a winning start.
Barry Cuddihy and David Goodwillie had Danny Lennon's side two up before the midway point of the first half and Ross Cunningham quickly restored their two-goal lead just before the break after Chris Hamilton had puled one back.
Rob Jones scored early in the second half but Clyde held on and on-loan Hearts defender Hamilton was sent off in stoppage-time.
Annan are top of Group F after following up their shock win over Hamilton with a 1-1 derby draw with Stranraer.
Ruari Paton opened the scoring for the visitors at Galabank but Owen Moxon levelled 10 minutes into the second half. Stranraer edged a penalty shoot-out 5-4 to claim a bonus point.
Joe Cardle scored a second-half double as Partick Thistle recorded their first win as they beat Queen's Park 2-0 in a Glasgow derby at Firhill.
Stenhousemuir claimed two points from a 2-2 draw against Edinburgh City at Ainslie Park despite trailing with five minutes left.
Botti Biabi gave Stenny an early lead but Blair Henderson levelled two minutes before the break and on-loan Livingston man Raffa De Vita put the hosts in front.
Callum Tapping levelled and the visitors won 4-2 on penalties.