The Greek champions, who had been looking to add to their opening day victory over Atletico Madrid, were on the front foot early on with Pajtim Kasami bringing a stop out of Robin Olsen before Kostas Mitroglou dragged an effort wide.
Giannis Maniatis then brought another save out of Malmo's Danish keeper as Olympiacos threatened to make a first-half breakthrough.
Mitroglou thought he had given his side the lead when he steered a Maniatis pass into the back of the net, only to see the referee's assistant penalise the Fulham loanee for offside.
Malmo then quickly broke out of defence and, within seconds of picking the ball out of their own net, Rosenberg took advantage of poor defending from Alberto Botia and keeper Alberto Jimenez Gago to tuck the ball into an empty goal.
After reaching half time ahead, the Swedish outfit appeared to be empowered and went in search of another goal with ex-West Bromwich Albion striker Rosenberg almost scoring a second as his shot from the left was saved.
On 82 minutes, Rosenberg secured victory after latching on to Pawel Cibicki's delivery into the area before slipping his shot into the top corner of Jimenez Gago's net.
Malmo's victory sees all four sides in Group A level on three points after Juventus lost to Atletico.