The future of Villa boss Alex McLeish is currently in doubt as the club faces a relegation battle going into the final few games of the season.
However, Lambert believes that it is not right for his fellow Scot's job to be discussed in such a way.
"It is extremely unfair with me being linked with Aston Villa when they have got a really good manager up there who is doing everything he can to stay in the league," Lambert told reporters. "It is an unfair comment for me to say anything on that."
The Canaries host Liverpool in the Premier League on Saturday.