The Rossoneri claimed a 5-4 win in a thrilling triumph that maintained their unbeaten start to the Serie A season.
"We must be very, very happy, as this is an extraordinary group. Let's not forget where we are coming from," Inzaghi told Sky Sport Italia.
"I saw the moves that we tried out in training used to score great goals, but I also saw that we conceded on every chance that we allowed. It was a very open game, but I think there can be few doubts that Milan deserved to win.
"At the final whistle I was thinking it would've been a shame to not win a game we were always in control of. That fear set in that this team, which so badly needs confidence, could've dropped points in this situation."
Milan are next in action when they host Italian champions Juventus in Serie A next Saturday.