However, Connor insists that there was no intent from the defender.
"We set off with a gameplan and were trying to work towards that," Connor told the club's official website. "And the penalty decision turns the whole game on its head.
"Once he's given a penalty and then the red card it's an uphill battle from there on. It's two lads racing for the ball and then there's a tangle of legs and I don't think it's fair.
"There is no intention there from Sebastien to impede Theo Walcott and to suffer a red card and a penalty is very hard to take."
Bottom-placed Wolves are now nine points adrift of safety following the loss at Molineux.