The Gunners continued to enjoy much of the possession but found it difficult to create any meaningful opportunities.
The Greeks nearly took a shock lead 10 minutes before the half-time whistle, but Ioannis Maniatis somehow put his effort over the bar from six yards after sneaking in between the Arsenal defence.
With the hosts thinking that they had a half-time lead, Olympiakos got level in injury time when Konstantinos Mitroglou glanced his header past Vito Mannone from Leandro Greco's cross.
Cazorla should have put them ahead after the interval when he sidefooted wide from close range following good work from Gervinho.
Mitroglou then fizzed a speculative effort narrowly wide before Lukas Podolski restored his side's lead.
The German capitalised on some persistent work from Gervinho before seeing his low shot creep under Megyeri.
Olivier Giroud missed two good chances to extend the advantage after coming off the bench, but fellow substitute Aaron Ramsey made sure of the win in injury time when he calmly slotted home.