The Black Cats were fined by the Premier League in December for playing Ji without international clearance.
The South Korean forward featured in four league fixtures and one Capital One Cup tie while ineligible.
"I don't think it will affect the team," Poyet said in Friday's press conference. "When too many things happen around a club, sooner or later it does affect the team, I believe that.
"This one in particular, I don't think it should affect us. It is nothing to do with how we are playing or the games we have got left.
"I know people think it shouldn't but it does. Now I don't think we have too many problems. But this has come as a surprise. People now maybe know why I couldn't play him. I knew, you didn't."
Sunderland, who have escaped a points deduction, have denied any wrongdoing on their part.