Matthews started a melee during his side's defeat to the 49ers after he hit Colin Kaepernick late, which resulted in him and Staley squaring up to each other.
Harbaugh told reporters: "I looked at it with my own eyes, I could see two punches thrown to Joe's head. Well, one punch and one open slap.
"If you're going to go to the face, come with some knuckles, you know, not an open slap. I think that young man works hard on being a tough guy, he'll have some repairing to do to his image after the slap."
San Francisco face the Seattle Seahawks next week.