Last season's Champions League winners went down 3-1 to Guy Poyet's side at the American Express Community Stadium despite taking the lead.
It was the Blues' third successive defeat in pre-season, and Di Matteo was not reluctant to hide his frustration.
"I was expecting more from my team in terms of urgency and movement," the Italian is quoted as saying on Chelsea's official website.
"I was looking for a better performance than the one we gave today. We were too stretched, never aggressive enough and didn't get the ball back quickly enough when we lost it.
"With the ball, there wasn't enough movement and there were a lot of things we didn't do. We have to be a bit more focused and sharper."
Chelsea will take on Manchester City in the Community Shield at Villa Park next Sunday.