The double from the midfielder represented Palace's first home league goals of the season, while Arsenal's 11-match winning streak in all competitions has now come to an end.
Both sides started brightly in the all-London showdown without creating much in front of goal, but Palace almost found an opener when they did get into a shooting position after 15 minutes.
Arsenal were offering little at the other end with their only chance of the first half coming when Alexandre Lacazette fired wide from 20 yards, although Hector Bellerin could have done better when being closed down in front of goal.
Palace earned five corners during the opening 45 minutes and the Eagles earned a chance to open the scoring from the final set piece after Mustafi dived into a challenge with Cheikhou Kouyate and only made contact with the midfielder.
Milivojevic had missed from 12 yards against Everton last weekend, but the Serbian made no mistake on this occasion as he rolled a low effort just inside Bernd Leno's right-hand post.
Arsenal needed to improve after the restart and, after a lively opening to the second half, Xhaka netted a stunning equaliser for his side as his free kick from the corner of the penalty area found the far top corner via the inside of the post.
The visitors remained in the ascendancy and it took just five more minutes to complete the turnaround, although their second came in controversial circumstances.
An inswinging corner was flicked towards the back post by the hand of Lacazette, and Aubameyang was waiting all on his own to prod the ball just over the line.
Palace found themselves on the back foot and struggling to create anything in the final third, but a cross from an acute angle from substitute Max Meyer struck the outside of the post and went to safety.
Mesut Ozil was substituted midway through the second half and the German threw his gloves into the dugout after an ineffective display, and Unai Emery's mood would not have improved when his side conceded a second penalty of the afternoon.
Xhaka dangled out a leg when Zaha opted to take him on in the box and, from the resulting spot kick, Milivojevic powerfully lashed a strike past Leno for his second of the match.
Both teams rallied as they searched for a fifth goal of the game, but a draw was probably a fair result after a clash which included a number of important decisions which will be contested by both managers.
CRYSTAL PALACE (4-3-3): Hennessey; Wan-Bissaka, Sakho, Tomkins, Van Aanholt; Milivojevic, Kouyate (Puncheon 88'), McArthur (Meyer 61'); Townsend, Ayew (Sorloth 64'), Zaha
ARSENAL (4-2-3-1): Leno; Bellerin (Lichtsteiner 46'), Mustafi, Holding, Xhaka; Torreira, Guendouzi; Iwobi, Ozil (Welbeck 68'), Aubameyang (Ramsey 78'); Lacazette