The Villans are currently two points behind leaders Chelsea in the Premier League table thanks to three wins and a draw from their opening four games.
While insisting that it is important for the squad to enjoy their recent success, Guzan is eager for Villa to keep improving.
"There's a lot of football ahead, but you have to enjoy the present - you have to enjoy the good things and good moments in football," he told the club's official website.
"Things are going well. We will enjoy it, but we accept the challenge of trying to get better week-in and week-out and trying to improve. That's what we will do."
Villa continue their Premier League campaign on Saturday when they host Arsenal.