Harrison hasn't stepped in the ring since his 2010 loss to David Haye and Adams believes that he can force the 40-year-old into retirement.
Adams said: "Audley is a joke. His fight with Haye was an embarrassment and he still wants to make a comeback.
"I will give him such a beating that Audley will see he does not belong in a boxing ring again."
The fight will take place at the Brentwood Centre.