Jabo Ibehre scored the only goal of the game in the 57th minute, but the home side did hit the bar in the first half.
The win means that the U's are now five points clear of the League One relegation zone, and Dunne says that his players must push on and secure survival.
"There's still 10 games to go but we have to win more games. There was a real desire to win the game and the team deserve it and so do the staff," Dunne told The Gazette.
"It's a win against a team who could have closed the gap on us and now we want to get some more. It was a real team performance - I thought we dominated the game and we made some good chances."
Colchester play Bristol City on Saturday.