The Buccaneers scored 15 unanswered points before the Dolphins finally got on the scoreboard late in the second quarter with a Rishard Matthews touchdown.
Miami then produced a third-quarter comeback to take the lead before a Bobby Rainey touchdown early in the fourth quarter provided Tampa Bay with what proved to be the winning score as they held out for a 22-19 victory.
The Buccaneers blew a 21-point lead against the Seattle Seahawks eight days earlier, and Hayward admitted that he feared they would suffer at the hands of another comeback yesterday.
"It was rough. It was scary, I'm not going to lie. It looked like last week. The guys, like Lavonte [David] and a bunch of us just said we're tired of it," Hayward told The AP.
"We can't do it again, and everybody just started stepping up and making plays."
The victory was the Buccaneers' first of the season and they now sit just one win behind the Atlanta Falcons in the NFC South.