The Gunners trailed 3-2 heading into the final 10 minutes of the game, but Aaron Ramsey drew the hosts level before Giroud struck the winner with just five minutes left.
Giroud insists that Arsenal's belief in their tactics and refusal to let their heads drop was crucial in turning the game around and claiming all three points.
The France international told Sky Sports News after the game: "Obviously I knew that I was going to have a chance so we tried to push.
"We kept the faith and kept believing in our game to push hard. We've been courageous to the end and thank God we won the game."
Arsenal's opening-day win is just their second in their last seven attempts.