Cole, 33, had been on trial with the Shrimpers at the start of last week, but Brown has revealed that the former West Ham United forward has now "disappeared off the face of the earth".
"Being completely honest about the situation Carlton seems to have disappeared off the face of the earth and I'm not really too sure what's happened," Brown told the Southend Echo.
"I've not been able to speak to Carlton or his agents. I would imagine he's got something lined up elsewhere now but it won't be at Southend United."
Cole, who won seven caps for England between 2009 and 2010, has represented Celtic, Sacramento Republic and Indonesian side Persib Bandung since re-joining West Ham in 2013.