After moving up two weight divisions to fight Gennady Golovkin, Brook is dropping back down to take on his mandatory challenger at 147lbs at Bramall Lane on May 27.
However, Mayweather has questioned Brook's decision to take a high-profile fight after being involved in five hard-hitting rounds against one of the biggest punchers in the world.
The American told Shady Slim: "I think him and Kell Brook is a good match-up but I don't think it's going the distance.
"I think Brook took a lot and it was a tough fight between him and Triple G, so I think, coming on the back of that, it's going to be a tough fight.
"It's not really fast but hopping into another tough fight like that, right after the Triple G fight? It's going to be rough."
Brook has just the one defeat on his record in 37 bouts as a professional, while Spence is unbeaten in 21 outings in the paid ranks.