The 52-year-old is preparing his side to face the Saints this weekend, three years after he was shown the door at St Mary's, which eventually led to his appointment at the North-East outfit.
"That's the managerial career," Pardew said in a press conference. "We're always vulnerable. I was vulnerable on day one, and I'm vulnerable now.
"I felt it was an injustice to lose my job. I'm not the first manager to say that. I always felt it was not a football decision. The fans were brilliant there. The players were with me, but the decision was made, and there was nothing I could do about it."
Newcastle are currently seventh in the Premier League table, three points above Southampton.