The Gunners won their first piece of silverware in nine years as they beat Hull City 3-2 after extra time at Wembley Stadium.
Despite insisting that the triumph should be celebrated, Gazidis is eager to start working on what he hopes to be a successful future for the club.
"We have a good platform to build on - today's a great day for us to celebrate and we should do that as the Arsenal family, but tomorrow we start work again and we've got to use this to progress the club, to take it forward," he told Arsenal's official website.
"This was a very big step, to have the players understand what it takes to come through that kind of adversity, to get over the line whatever it took is a big step and now we need to take the next step."
Saturday's win saw Arsenal triumph in the FA Cup for a record-equalling 11th time.