Michael Laudrup's team lost 2-1 to West Bromwich Albion on Saturday, and Williams has admitted his side took their foot off the pedal.
"Looking at our performance as a team we could have done better," he told reporters.
"It's gone, we need to talk about our performances. When we scored, we stopped playing for some reason. We stopped attacking them, they're a good side and a team like that you have to finish off.
"That's the frustrating thing about this game. We did brilliantly last week against Newcastle. This time I felt we took our foot off the pedal and let them back into it."
Swansea currently sit in ninth place in the Premier League table.