The home side got off to the perfect start when Dominic Oduro turned the ball into the net from Jairo Arrieta's cross.
United thought that they had got back into the game almost immediately when Lionard Pajoy headed the ball past Andy Gruenebaum only for the goal to be ruled out for offside.
The visitors again went agonisingly close to getting a goal back but Kyle Porter's shot was turned onto the post Gruenebaum.
It got even better for the hosts just before the break when Arrieta was fouled in the area by Brandon McDonald and they were awarded a penalty. Federico Higuain made no mistake from the spot.
After half time, chances were few and far between and United had a good opportunity to get one back but Pajoy was denied by Gruenebaum, who made a solid save with his feet.
United pushed to get something out of the game but Columbus held onto their lead comfortably to seal the three points.