The 24-year-old scored his second goal for the club on Saturday in a 2-0 win over Cardiff City.
Kozak now believes that Villa can record a second straight victory by beating the Baggies at The Hawthorns.
"It is another important game after the international break," Sky Sports News quotes the Czech as saying.
"We must concentrate on that game. West Bromwich will be a hard adversary, but if we play like we did against Cardiff I think we can win.
"It's my first derby and I'm excited for this game. For everybody it is important."
Villa travel to West Brom in the Premier League on November 25.