© PA Photos
An evenly-matched opening spell ended with a Yann Kermorgant goal in the 19th minute, as the French striker netted against his former club.
The Foxes made a bright start to the second half as they pushed for an equaliser, with Knockaert, James and Chris Wood all looking sharp.
However, Michael Morrison had a chance to double the visitors' advantage in the 50th minute when Johnnie Jackson fired a free kick in to the Leicester area, but Morrison's shot went straight to Kasper Schmeichel.
The home side were not level for long though, as Haynes put the visitors back in front in the 78th minute, having been on the pitch for just over a minute.
Leicester kept looking for another equaliser, with Wood and Danny Drinkwater both pressing hard, but the hosts could not get level before the final whistle.