The Colts came into the clash in fine recent form and were first to get off the mark, shading a scrappy first quarter thanks to Adam Vinatieri's 30-yard field goal.
Rivers and Allen combined again soon after for the game's first touchdown, with a low dive turning a 74-yard drive into a home score.
San Diego were soon back in touchdown range thanks to variety on offence, but the Colts showed firm resistance to only concede a field goal, which was coolly converted by Novak.
Indianapolis had just under two minutes to respond, but with Andrew Luck let down by receivers the Colts were forced to settle for a 50-yard field goal attempt.
Vinatieri converted from range to keep the Colts close after a tough second quarter, despite their struggles on offence.
Novak soon replied after the break to keep the Chargers safely ahead, with some strong running from running back Ryan Mathews continuing to create field-goal opportunities as he drove past the 100-yard mark.
Early in the fourth quarter Luck was sacked as the Colts struggled on offence, with Novak's trusty boot taking full advantage to move San Diego seven points ahead.
The Chargers then looked to extend their lead late on as Antonio Gates came up short of a big first down by inches, with Novak asked to kick a crucial 50-yard field goal as a result.
With the pressure on Novak delivered, drilling down the middle to put the Chargers in total control with under two minutes left.
Luck's desperate late attempt to respond resulted in an interception, leaving the Colts to lick their wounds after a second loss of the season, with the Chargers improving to a 3-3 record.