The Uruguayan takes charge of the Black Cats for the first time on Saturday following his appointment last week.
Sunderland head into the match at the Liberty Stadium at the bottom of the table with one point from seven matches.
"Swansea are in a better balanced position, yes they are, it's true," Poyet told the Sunderland Echo. "But that doesn't mean that they are going to win.
"We will go there to win. We need to win football games. We cannot go to places looking to get just one point."
Sunderland have lost their last five matches in the Premier League.