Sherwood has seen his promising start to life as Spurs boss come to a shuddering halt with four defeats in his last six league games after losing just one of his first 10 in charge.
The 45-year-old has also received criticism for his decision to watch matches from the stands rather than on the touchline, but he was quick to defend himself and insisted that he knew things would get tougher eventually.
"You are damned if you do and damned if you don't. If Jose Mourinho sits up there, he's fine. Nigel Pearson at Leicester has sat up there all season. But I am realistic to know it's Tottenham, you are under the spotlight no matter what you do," Sherwood told reporters.
"You've got a rookie manager coming in here, won the first eight games or whatever it was and everyone thinks it's easy. I didn't, I was realistic, I knew there were going to be some downs.
"It was a case of just cracking on. But whatever decisions I make are my decisions. If I'm going to go down, I'll go down my way."