Alex Iwobi put Arsenal into a third-minute lead, but a long-range strike from Benjamin Bourigeaud, an own goal from Nacho Monreal and a late header from Ismaila Sarr have given Rennes a commanding advantage.
The pendulum of the opening game of the last-16 tie swung in Rennes' favour just before half time when Sokratis Papastathopoulos was sent off for two bookable offences, and the Gunners will now have to come from behind in next week's return fixture at the Emirates Stadium.
Arsenal enjoyed the perfect start but their early goal came in fortuitous circumstances, with Iwobi's inswinging cross from the left evading everyone and going into the net via the back post.
Rennes responded positively after going behind as they began to up the tempo on the ball but Arsenal should have doubled their advantage before the quarter-hour mark, with Pierre-Emerick Aubameyang seeing a first-time effort from 12 yards punched away by Tomas Koubek.
Clement Grenier sent a 35-yard free kick for the home side whistling over Petr Cech's crossbar, and Arsenal responded by going down the other end with Aubameyang having a low attempt comfortably dealt with by Koubek at his near post.
Arsenal went close again after 28 minutes as Shkodran Mustafi fired agonisingly wide of the far post after a well-worked free kick, but Rennes had the better of the closing stages of the first half.
Hatem Ben Arfa saw a set piece punched away by Cech after Sokratis had picked up the first of his two yellow cards, and his second caution came soon afterwards when the centre-back was adjudged to have clipped Sarr on the edge of the box.
Bourigeaud took full advantage from the resulting free kick, with the midfielder sending his first effort into the wall before finding the far top corner of Cech's net from 25 yards out on the rebound.
Rennes wasted no time in going on the attack after the restart, and Bourigeaud should have put the hosts in front instead of only picking out Cech with a first-time effort from 12 yards.
Unai Emery acted by introducing Matteo Guendouzi for Iwobi but the substitution did nothing to stem the tide, and Cech had to make another save down to his left to keep out Sarr's strike from just inside the box.
Ben Arfa then had a free kick tipped over the crossbar by the Czech goalkeeper, but there was little that the veteran could do to prevent Monreal from putting through his own net.
A ball was delivered into the area from the byline, and all Monreal could do was divert the cross into the far corner of the net.
Arsenal should have equalised soon afterwards only for Mesut Ozil to fire straight at Koubek after creating an opportunity for himself inside the area, but Rennes remained on the attack during the closing quarter.
Emery introduced Sead Kolasniac for Aubameyang in order to try to stay within a one-goal deficit ahead of the game in North London, but Rennes netted a potentially tie-changing third just before full time.
A cross from the left was perfect for Sarr who was waiting at the back post, and Rennes' star performer on the night powered a header past Cech to seal one of the biggest results in the history of his club.
RENNES (4-4-1-1): Koubek; Zeffane, Da Silva, Mexer, Bensebiani; Sarr, Andre, Grenier (Gelin 90+1'), Bourigeaud (Lea Siliki 73'); Ben Arfa; Hunou
ARSENAL (4-2-3-1): Cech; Mustafi, Papastathopoulos, Koscielny, Monreal; Torreira, Xhaka; Mkhitaryan, Ozil (Ramsey 70'), Iwobi (Guendouzi 53'), Aubameyang (Kolasinac 79')