The 37-year-old has been accused of using "inappropriate language" in two incidents allegedly involving John Obi Mikel and Juan Mata during Chelsea's controversial 3-2 defeat to Manchester United at Stamford Bridge.
"And faced with the very serious allegations he will also feel angry and worried," Poll wrote in the Daily Mail.
"Angry that he is subject to such allegations given all the abuse that he, along with so many other officials, allow themselves to be subjected to week in week out; and worried in case, in the heat of the moment he forgot himself and made inappropriate comments.
"Not racist ones, he'll know if he made those and will know his career is over if he did; but comments including foul or inappropriate words that he cannot recall making. When the pressure is at its greatest things are said and done which do not bear scrutiny."
The Metropolitan Police is investigating whether Clattenburg used any racist language towards the Chelsea players.