Lukaku is currently on loan at the Baggies from Chelsea, and it is unclear whether the Belgian forward will return to Stamford Bridge next season to fight for a place in their first team.
The 19-year-old is likely to attract many suitors should he be allowed to leave Chelsea for a further 12 months, but Clarke does not think that is next destination will be the Liberty Stadium.
The 49-year-old told reporters: "There is nothing in it. It is idle gossip - it's lazy.
"I read in the press that he's going to Swansea - but I would be very surprised if he ends up there."
The former Anderlecht striker has scored 13 goals for West Brom this season.