The Swans conceded twice in the opening 21 minutes to go down 2-1 at home in the Premier League.
"They had the better start, especially Gareth Bale - he made the difference," he told BBC Sport. "We came out after half time and dominated the second half.
"I'm a little disappointed, not with the performance or attitude of the team, but this game was a perfect opportunity to beat Tottenham. After an international break, you won't get a better opportunity.
"It's important for us now, we have seven games to go and we should go for the points, we won't get ninth or 10th with 40 points that's for sure."
Swansea are currently ninth in the league table.