The Gunners will defend a 3-1 lead at the Emirates Stadium next week and were good value for their win in Greece.
Martin Odegaard's first Arsenal goal had them ahead only for another defensive calamity to gift Youssef El-Arabi the equaliser just before the hour.
Mikel Arteta saw his side dumped out by Olympiacos at the round of 32 stage last season despite winning the opening leg in Greece but will be hoping lightning does not strike twice.
Arsenal were back at the Georgios Karaiskakis Stadium for the second time in a fortnight, having hosted the 'home' leg of their tie against Benfica on the outskirts of Athens due to current coronavirus travel restrictions.
The Premier League side have now won more Europa League knockout stages at the home of Olympiacos than the reigning Greek champions themselves – but they made it harder for themselves than was necessary.
Odegaard was again deployed as Arteta's number 10 but he missed a fine early chance to put the visitors ahead, scuffing a shot wide after being picked out by Kieran Tierney.
If the Norway international was guilty of a poor miss, Pierre-Emerick Aubameyang was denied soon after by a fine Jose Sa save as the Olympiacos goalkeeper tipped his header onto the crossbar.
The hosts had started slowly but were beginning to edge into the contest and Bruma should have done better when gifted possession by Odegaard, the winger only able to shoot straight at Bernd Leno.
It was a number of sloppy moments for the Real Madrid loanee but he would make amends by thrashing Arsenal in front.
Thomas Partey played the ball into Odegaard in a central position and his arrowed shot flew in off the wrist of Sa – who would have been disappointed not to keep out the strike.
Olympiacos were almost level as Arsenal were again caught trying to play out from the back, David Luiz's pass being closed down by Giorgos Masouras, who then shot wildly wide of goal.
That was just a hint of what was to come as Arteta's men would all-but present Olympiacos with their equaliser.
Just like at Burnley on Sunday, Leno played the ball short and into the feet of Dani Ceballos – fresh off the bench.
He was closed down quickly, losing possession as El-Arabi levelled with a crisp finish past an out-of-position Leno.
El-Arabi's last-gasp winner sent Arsenal out of the competition last season and he almost doubled up for the night, only to see a shot blocked behind when well placed.
Arsenal regathered their collective poise and again began to dominate proceedings as their former defender Sokratis Papastathopoulos – deemed not good enough to even be registered by the club before leaving for Greece in January – led the defensive drive for the home side.
But they could not hold on to take a draw to London next Thursday as Willian collected a short corner and crossed for Gabriel to send a looping header over Sa to re-establish the lead.
Elneny came off the bench after Gabriel's effort and his first involvement was to curl home a low effort from outside the box and put his side firmly in the driving seat and send them into Sunday's north London derby against Tottenham with a win.