Mark Clattenburg was the centre of attention at Stamford Bridge as Branislav Ivanovic and Fernando Torres were sent off for the hosts before Javier Hernandez scored the winner from what looked like an offside position.
"The referee's decisions influenced the result," said Di Matteo. "The sending-off of Fernando Torres and the third goal is offside so that clearly has decided the game for our opposition tonight.
"The referee is probably going to watch it again and realise he made mistakes. But that is not going to change the result today."
Despite their first Premier League loss of the season, Chelsea remain top of the table.