While already on a yellow card, Yedlin lunged into a challenge with Nathan Redmond and made contact with the winger, only for the official to give the full-back a final warning rather than a second caution.
Southampton were still able to fight back to earn a 2-2 draw, but the ex-official has acknowledged that Kevin Friend had made a poor decision.
Gallagher told Sky Sports News: "Whether it is a straight red is up for debate. What isn't up for debate is that it is a second yellow card.
"I don't see the purpose of bringing in the captain (Jamaal Lascelles) because the purpose there is to prevent it happening again. He should never be allowed to do that again.
"The easiest decision is a second yellow card. We wouldn't even be questioning the decision."
Gallagher also claimed that Friend was correct in awarding Southampton a second-half penalty after Shane Long was upended by Florian Lejeune.