The home side found the endzone on their second possession of the game, with Doug Martin getting the Buccaneers into a strong field possession with a series of runs, before Jameis Winston kept the ball, running over in the corner.
Julio Jones moved the Falcons deep into the Tampa Bay half as Atlanta looked to respond, but Matt Ryan could not get the ball to Roddy White for the touchdown, before the quarterback was sacked on the next play, leaving Graham to kick the field goal.
Vincent Jackson made a 36-yard reception as the Buccaneers threatened to extend their lead further in the second quarter, but the drive stalled, forcing the home side to punt.
Ryan was sacked twice after the Atlanta offense came back onto the field, but Devonta Freeman, who missed last week with a concussion, got the Falcons in field-goal range again, allowing Graham to cut the lead to a single point just before half time.
The Falcons got the ball back again inside the final minute of the first half as Winston threw an interception, giving Graham his third field goal of the quarter, with the kicker splitting the posts from 52 yards out.