The Swans led 3-2 with just seconds remaining at the Liberty Stadium before Charlie Adam earned the visitors a share of the spoils from the penalty spot following an alleged handball by Wayne Routledge.
However, Williams is convinced that referee Robert Madley was wrong to punish Routledge.
"It seemed like none of their players asked for a penalty, maybe one of them was appealing for a corner," Sky Sports News quotes the defender as saying.
"It was a shock to everyone and I didn't see a handball from Wayne or Leon [Britton]. I think both sides were surprised by the decision."
Swansea have the chance to get back to winning ways in the Premier League on November 23 when they make the trip to strugglers Fulham.