Having lost Carl Jenkinson to a red card in the second half, the visiting Gunners came under a large amount of pressure from their hosts.
However, some last-ditch tackling and saves from goalkeeper Szczesny ensured that Arsenal travelled back to London with all three points.
"I thought we deserved the win and the urgency in our own penalty area was fantastic," Szczesny told the club's official website. "Bacary Sagna came in at centre back and cleared everything that came into the box. It was difficult once Carl got sent off, and we were playing with a back four that had never played together. But I thought they all did a tremendous job.
"Every ball that was played across, we were the first to clear it, so they deserve a lot of credit for the way we played.
"I think we proved to a lot of people that we have what it takes to be successful because it wasn't the best game in terms of quality, but we showed everyone the character and desire we have to get a result."
The win has left Arsenal fifth in the Premier League table, four points adrift of rivals Tottenham Hotspur in fourth.