With both sides playing out a 0-0 draw after extra time, AJA were triumphant from the spot as they went through 4-2.
© Getty Images
Brest then enjoyed a bright spell, but the opposition came back at them in the 37th minute, when Samed Kılıc surged down the right before blasting a shot into the arms of the goalkeeper.
Neither side could break the deadlock by half time, and chances continued to be at a premium well into the second half.
In the latter stages of the game, Baby and Sammaritano threatened on a number of occasions for the visitors, but come the end of normal time, it was still goalless.
Auxerre continued to be on top during extra time, with Vincent Gragnic at the heart of most of the action, but Brest were able to hold on for the full 120 minutes.
Youssef Adnane and Simon Falette missed their spot kicks for the hosts, and despite Thomas Fontaine's failed effort, Sammaritano converted his side's final effort to put AJA into the last four of the competition.