Reports have suggested that the two had fallen out earlier this summer, which had resulted in the Togo frontman training away from the senior side.
However, the 29-year-old has insisted that it was the death of his brother that forced him to take some time out, rather than a disagreement with Villas-Boas.
"In the life of every footballer there are always ups and downs but I'm not complaining. It all started well in pre-season. We flew to Hong Kong and there I learned the news of the death of my brother, so my morale took a hit. I went back for the funeral. I had a very difficult time," the London Evening Standard quotes Adebayor as saying.
"When I went back to Tottenham, I talked to the coach and he told me that I was not ready yet for the pro group. I had to go to work in the reserves for two to three weeks to regain my confidence, my sense of goal and I complied with his advice.
"Was it a punishment? Not at all. I'm not at a training centre to get punished by my coach. There has been no argument. I have been playing professional football for 11 years and my brother's death was a blow to me in recent weeks but since then I found my spirits. I will find my feelings and my place in the group."
Adebayor joined Spurs on loan from Manchester City in 2011, before making the move permanent a year later.