Spurs boss Andre Villas-Boas will face the Blues for the first time since losing his job at Stamford Bridge after just nine months in charge.
"It is going to be an important game because the table is so tight," Dempsey told Sky Sports News. "You have to take care of your home games and try and get as many results as you can.
"We want to continue this run of positive results, so we know it is going to be a tough test but we are all excited, and we have everyone back from fitness now and we are looking to make the most of this game."
Tottenham currently sit fifth in the league table while Chelsea remain unbeaten at the top.