The former Premier League champions came from a 2-0 deficit to draw level at Anfield, but a mistake from Kompany allowed Philippe Coutinho to latch on to the ball and clinch a 3-2 win for Liverpool.
"Obviously he is disappointed, everybody is," the Manchester Evening News quotes Milner as saying. "You can't blame Vinny, everybody makes mistakes.
"Vinny has been outstanding for us. I've been at the club for four years and he's been absolutely outstanding. You can't say anything like that because of the amount of times he's saved us as well - it's one of those things."
City are third in the table, seven points behind leaders Liverpool with two games in hand.