The Wolves were beaten at home for the first time this season as the visitors scored four tries at the Halliwell Jones Stadium.
"I don't know if we lost it but I think a lot of damage was done in the first 20 minutes," Smith told Sky Sports News. "We made way too many errors - some forced, but some unforced.
"Saints were terrific in their defence and that rattled us at times instead of having some patience and sticking with it."
The defeat has left Warrington in fourth place in the Super League table.