The Bolton fighter, who squares off against Diaz in Sheffield on April 27, thinks that a loss to the Mexican would be disastrous for a career that has already seen him lose two of his last three fights.
"I can't afford to lose again - I'd be destroyed," the Daily Mail quotes Khan as saying.
"I needed someone like [new trainer] Virgil [Hunter] to tell me straight how it is. If I had stayed in Los Angeles, I could have become a big-time Charlie."
The 26-year-old also confessed that training for fights out in the glamorous Los Angeles did little to help him prepare fully for fights, and said that his head was turned by the number of famous names who came to watch him.
"Stars like Mark Wahlberg, Mickey Rourke, Snoop Dogg, Sugar Ray Leonard, even Lennox Lewis, came to see me train. You think you've made it. I used to put on a performance. They want to be your friend, but I now know it takes away from what got you there in the first place: boxing," he said.
"Normally, I'm a very humble man. But when people come over and say there's this party happening and people like yourself should be there, it's a big thing, it does change you."
Khan's brother Haroon will make his professional debut on the undercard of the Motorpoint Arena show.