The hosts made the worst start possible after just four minutes when Andrea Rispoli gave away a penalty. Massimo Loviso stepped up to take the spot kick, and fired past Palermo keeper Simone Colombi to give Alessandria an early lead.
Palermo were reduced to 10 men on 22 minutes, as Franco Vazquez received a red card for violent conduct, and a Michele Marconi volleyed goal just one minute later appeared to end the match as a contest.
The Rosanero could have pulled a goal back before the break when Aleksandar Trajkovski found himself in a good position, only for Alessandria defender Alex Sirri to make a last-ditch tackle and deny the hosts.
Palermo began the second half positively, and pulled a goal back on 55 minutes when Robin Quaison set up Trajkovski for a composed finish past Alessandria keeper Gianmarco Vannucchi.
Both sides came agonisingly close to adding to the scoreline moments after, as Trajkovski hit the woodwork for Palermo, before Marconi saw a goal disallowed for offside.
Alessandria extended their lead on 82 minutes thanks to a solo effort from midfielder Gianluca Nicco, edging the Piedmont side closer to the fifth round of the Coppa Italia.
The home fans were given renewed hope of a comeback late on, as Vannucchi parried a Trajkovski effort into the path of substitute Alberto Gilardino, who slotted home for his side's second.
The visitors held on though to claim a shock 3-2 win and progress to the fifth round.