The 24-year-old courted interest from the Londoners under Carlo Ancelotti's reign at the club but, with the Blues boasting stellar names such as John Terry, Ricardo Carvalho, Branislav Ivanovic and Alex at the time, the Croat opted against a move in favour of Lyon.
The Saints travel to Stamford Bridge this afternoon knowing that victory would take them above Jose Mourinho's men in the Premier League, and Lovren has no regrets over rejecting the club's overtures four years ago.
He told The Sun: "It was my decision. I thought it might have been easier for me to play at Lyon at that time. Chelsea had John Terry, Ricardo Carvalho and Alex. I was only 20 years old.
"I took the chance because I wanted to improve myself. I think it was the right decision and it has worked out well.
"If I signed for Chelsea, maybe I would have sat on the bench. What would be the point?"
Today's clash in the English capital is scheduled to kick off at 4.10pm.