Blackwell insisted that his side were the better team until giving away possession in the build-up to Colchester's opener.
"Fair play to Colchester - they dug in when they needed to and they got the clean sheet as well," Sky Sports News quotes Blackwell as saying. "Their keeper made two good saves and came and got some crucial crosses.
"We came out after half-time as the dominant side, but we gave them their first goal after we had possession of the ball.
"There's no need to do what he's done and we gave away a stupid goal and that gave them a lift, which was all that they needed. It just shows you that if you play in the wrong areas, that's exactly what can happen."
Bury are currently second from bottom of League One.