Pardew's men could face a trip to Chelsea in the Premier League just two days after playing the Greek side in their Europa League fourth qualifying round tie.
"It'll be a hard game, for sure," Pardew told the club's official website.
"You don't finish fourth in your league unless you've got something going on so they're obviously on the up.
"It's a tough journey which is going to take a little bit away from us. Certainly coming back from there to go to Chelsea on a Saturday is not ideal."
Atromitos came fourth last season in the Greek Super League, finishing 23 points behind champions Olympiakos.