Fury had claimed that he would end Haye's career should the two meet in the ring, which prompted the 32-year-old to propose a bout between the two Brits.
However, after Haye's trainer attempted to open negotiations with Fury's promoter, Mick Hennessy, he was quickly rejected.
Haye told ESPN: "Adam contacted Mick Hennessy and proposed the fight. They flat out said 'no'.
"Hopefully he keeps winning because he looked pretty shaky in his last fight."
The two-weight world champion will make his comeback to the sport in a bout with Manuel Charr on June 29.