The 48-year-old, under contract with the Blues until the end of the 2018-19 season, has admitted in the past to feeling homesick in West London.
Speaking after the champions' 1-0 win over Southampton at the weekend, though, Conte claimed that the uncertainty being created in the press is possibly affecting his players.
"Every day there is speculation about my future," he told reporters. "It's not right because I think I deserve a bit of respect for what I did last season, for what I am doing this season.
"To read every week: 'Next season the club contacts this coach or this coach' - for me this is not a problem, but it can create problems with the players because the players believe we could [drift] apart.
"I have another year on my contract, so in this moment my focus is total, to improve my players, to try to improve the team, to try to improve the club. If you understand football, there is a moment you must have great patience, to know that someone is working very well.
"I think last season we did a fantastic job but this job is continuing this season. We are trying to change the average age of our team. I think for a club like Chelsea it is very important to start to do this, if you want to create a new base and face the future with confidence."
Conte won the Premier League title in his first season at Chelsea, doing so with a record number of wins along the way.