The Londoners received a world-record £86m following the sale of Bale to Real Madrid and they immediately re-invested that by signing seven new players.
However, the new arrivals, including Erik Lamela and Roberto Soldado, struggled during their first seasons at White Hart Lane and Sugar has put the blame solely on former manager Villas-Boas for recklessly buying players that were not good enough for the club.
"I sat back and watched that happen and it can't be right," The Mirror quotes him as saying. "It's right that you promise the manager the proceeds of the Bale sale and say he would be given all to spend on players. That's great.
"But I would guess if you had made that promise to Arsene Wenger then you would have gone, 'Thank you, Mr Chairman. That's good. Stick it in the bank and I will use it when I find somebody who I think is worthy to buy.'
"But they went out like a kid in the sweet shop. [Villas-Boas] bought anything in sight and went and spent the money straight away."
Sugar was chairman of Spurs from 1991 to 2001.