Jack Wilshere found the net in the final five minutes to set up a trip to Brighton & Hove Albion in the next round for Arsene Wenger's side.
© PA Photos
Only the woodwork kept Swansea out in the 25th minute when Kyle Bartley headed against the crossbar following Wayne Routledge's deep cross.
Despite a disappointing first half for Arsenal, they came alive in the final five minutes as Theo Walcott squandered two decent opportunities.
The England international had his first effort blocked by a heroic Swansea defence before a lack of communication with Wilshere saw the England international lose possession.
Walcott was again guilty of missing a glorious chance at the start of a thrilling second half when he flicked the ball the wrong side of the post with just the onrushing Michel Vorm to beat.
Graham then came to Swansea's rescue on the hour by clearing Walcott's drive off the line.
© PA Photos
Both sides went in search of a result in the final 10 minutes of normal time, with Olivier Giroud hitting his shot straight at Vorm before Chico Flores put the ball narrowly wide with a header.
The woodwork was struck again in the 82nd minute as Bacary Sagna's cross evaded everyone on its way to hitting the far post.
With the match seemingly heading for extra time, Wilshere powered his low shot home from the edge of the box to send Arsenal through.