The away team had an early penalty kick but Gareth Steenson was unable to convert his effort.
Steenson then hit the crossbar with another penalty effort and Gloucester were able to prevent Exeter from scoring a try from the rebound.
However, Steenson finally got the first points of the game from a penalty to put his side 3-0 up.
Exeter restored their slim advantage with Hoani Tui's penalty kick, but before that, Steenson was guilty of another miss.
The visitors moved further ahead through a Ben White try, and this time Steenson made no mistake with the conversion.
Midway through the second half, Dean Mumm extended the Chiefs' lead, crashing over after numerous carries from the Exeter forwards.
Gloucester forced their way back into the match thanks to a Jonny May try and Burns conversion, but Steenson edged the Chiefs further ahead with another two penalties.