The League Two outfit won the match courtesy of second-half goals from Conor Shaughnessy and Shawn McCoulsky, cancelling out Gaetano Berardi's early opener, and the Whites head coach conceded that his side should have performed better.
"I'm very disappointed. It's a team we should have beaten. We have a better squad. It doesn't matter how many changes we made for the match, we should have beaten a side from League Two," Christiansen told the club's official website.
"We didn't create enough opportunities in the second half and made all of the mistakes we were hoping to avoid," he added. "It's shame we're out of a nice competition like the FA Cup."
The result saw Newport book their place in round four of the competition for the first time since 1979.