Phillips scored the winning goal against his former side when playing for Birmingham City five years ago, and the veteran forward has indicated that he wouldn't repeat the act should he net against them for the Eagles on Saturday.
The 40-year-old told the Daily Star: "I had six great years with Saints as a teenager and two more after coming back from Sunderland.
"I came off the bench and got Birmingham's winner with a couple of minutes to go and it was just a reaction. I should not have celebrated that goal.
"I have so much fondness for Southampton. The fans and the club have always treated me brilliantly."
The former Sunderland frontman scored 26 goals in 73 appearances during his time at the Saints between 2003 and 2005.