The Baggies are just three points clear of the relegation zone in the top flight, but have taken five points from their last three games, which could have been nine if not for two last-minute goals against Tottenham Hotspur and Cardiff City.
Mulumbu told the club's official website: "We've done well against the big teams this season. It's a crucial game for Manchester City but also for West Brom as well. On Wednesday we saw Sunderland draw against City so anything is possible in football, especially in England.
"It's like a cup game. You never know what's going to happen. We just need to go there and start well, start sharp like we did last Saturday against Tottenham and show a little bit more strength at the end of the game."
West Brom are 16th in the Premier League with 33 points after 34 games.