The Belgian international has impressed while on loan at the Baggies this campaign, notching up 13 goals for Steve Clarke's side.
Olsson told reporters: "Romelu has been massive for us. It's not realistic to think we keep him.
"Someone who scores 13 goals for a club like West Brom, with all respect, it's a great achievement. He'll be difficult to replace but that's the challenge."
West Brom are currently eighth in the Premier League.