Matt Hasselbeck continued to perform well at quarterback for the Colts in the absence of the injured Andrew Luck as he moved his side into the range of Adam Vinatieri, who put the hosts ahead through a 49-yard field goal in the first quarter.
The Buccaneers rallied through the running of Doug Martin when he plowed through the line to move into Colts territory before the visitors were halted short of the endzone, but Connor Barth sent a field goal through the posts.
Indianapolis responded on their next possession and the boot of Vinatieri proved to be vital again as he was able to put his side ahead from 27 yards.
Lovie Smith's men managed to punch the ball into the endzone, but Martin saw his effort ruled out for a holding penalty. However, Barth levelled the scores with a field goal.
The Buccaneers took the lead for the first time in the match just before the break when Brate collected a powerful throw from Jameis Winston to find the endzone.