This weekend's clash has been overshadowed by the racism case involving the centre-back and Blues skipper John Terry, who will not be shaking hands with Ferdinand before the match after the decision was made to scrap the pre-match ritual.
"There is a big rivalry between the two clubs anyway but, with everything that has happened, I know there will be some fans targeting me," Ferdinand told the London Evening Standard.
"The Chelsea fans believe what they want to believe. They have their own view on what's gone on and if they are going to give me stick, that's their problem.
"All I can say is that it won't upset my game. If they think that's what's going to happen, then they have got another thing coming."
Terry is due to stand trial in July for allegedly racially abusing Ferdinand during a fixture between the two clubs last October.