However, two goals in the first eight minutes proved their downfall against the Premier League champions, and Sullivan told Sky Sports News: "I think all we can do after Saturday's defeat is say sorry to the supporters.
"To be 2-0 down after seven minutes of any football match, be it in the Premier League or in your local Sunday League, makes it very difficult to get a positive result.
"In parts of the second half we were very good. But, we need to be good before we are losing. We have to find out why we are conceding so many goals and fix it because it is obviously harder to win games when you concede two or three goals in a game than if you keep a clean sheet."
Up next for the East Londoners is a Premier League trip to Hull City on Saturday, April 1.