The Pride of the South tried to push for an early goal, but John Stegeman's side managed to hold out for the first half-hour.
However, Giovanni van Bronckhorst's men finally broke the deadlock in unusual circumstances after Eljero Elia managed to shake off his marker and his cross managed to evade everyone and bounce into the net off the post, despite the best efforts of Bram Castro to claw the ball away.
Feyenoord could have had a second goal on 36 minutes when Simon Gustafson's shot deflected off Castro and off the top of the crossbar.
With half time approaching, Heracles thought they had a good shout for a penalty after Oussama Tannane went down in the area, but the referee waved away his appeals.
As the visitors continued to frustrate Feyenoord, the Rotterdam club almost doubled their lead following Dirk Kuyt's free kick, which took a deflection before drifting narrowly wide of Castro's right-hand post.
Heracles's best chance of an equaliser came late on when Tannane latched onto a through ball and, with only Kenneth Vermeer to beat, he put his effort wide of the target.
Moments later, the hosts added a second after Kuyt netted his 11th goal of the season before Kramer sealed victory with a close-range effort in the last minute.