The Reds boss was allegedly on the end of some strong words from a home fan at the Liberty Stadium, which he took offence to midway through the second half.
Asked about the incident after the match, the German told reporters: "He was shouting at me all the time. Sorry, I reacted one time, I remind myself I am a human being and not a professional manager who takes that all the time and they can say whatever they want. At one point, I said 'please'.
"I had not a recorder like you so nobody taped it, nobody could hear what he said. That's fine. That's how it is. I have no problem with that. I just had one moment where I thought 'now, that's enough'.
"I'm not sure if we were already 1-0 down, I think we were, but it had nothing to do with it, but only enough. He felt quite good because nobody can do anything. He is in a good position. I'm sure I'm not the first manager [for this to happen to].
"I think he's a season-ticket holder. It's my fault as well, I shouldn't have reacted. But I didn't say anything bad, only that I showed I wasn't too happy by what he was saying all the time."
Against no side has Klopp lost more games against during his time in English football than Swansea (3).