Johnson left Manchester City for Martin O'Neill's side in a £10m move last month after becoming frustrated with the lack of playing time at the Premier League champions.
"It was difficult, it was tough," the 25-year-old is quoted as saying by The Mirror. "I was there or thereabouts with the last manager. He still called me up, apart from going to the Euros.
"But obviously sitting on the bench for any amount of time is going to hamper your chances if someone is playing ahead of you, and doing well week in, week out.
"If you're on the bench and not even playing, then I think it's only right that the other person plays, isn't it? It plays a little bit on your mind."
Johnson added that playing for England was an important factor in his decision to join Sunderland.
"The England manager (Roy Hodgson) has come out and said he wants his players to be playing regularly, that you can't sit on the bench forever," he said.
"So that's another reason for me to come to Sunderland and hopefully get more games and keep carrying on playing for England. Hopefully I'll be going to the World Cup in Brazil."
Johnson is part of the England squad currently preparing for their 2014 World Cup qualifiers against Moldova and Ukraine.