Rochdale got off to a blistering start, taking a 1-0 lead after just six minutes through a goal from Andrew Tutte.
The early goal shocked Barnsley, who took a while to recover as Barnsley continued to attack.
However, the away side managed to get back into the game and threatened Rochdale's goal on a few occasions.
Barnsley finally got an equaliser on the stroke of half-time, with a goal from John Stones.
The second half saw both sides press forward in a bid to take the lead. Barnsley looked the better of the two sides and had a couple of half chances that were just missed.
The away side's dominance paid off after 79 minutes when striker Craig Davies made it 2-1.
As the game headed towards the final whistle, Barnsley goalkeeper Ben Alnwick gave away a penalty and got himself sent off.
Rochdale's Jason Kennedy kept his cool and slotted the penalty in the bottom corner to make it 2-2 in the fourth minute of injury time, taking the game into extra time.
Barnsley were the first to make a breakthrough in extra time after Chris Dagnall made it 3-2 on 95 minutes.
Dagnall got his second goal on 105 minutes to make it 4-2 and seemingly secure their place in the second round.
However, the game was not finished as Rochdale won another penalty on 106 minutes, which was finished well by Kennedy once again.
The game finally came to an end with the score at 4-3, sending Barnsley into the second round.