Both sides wrestled for control throughout the first half, and it was Columbus who then came agonisingly close to breaking the deadlock in the 26th minute when Tony Tchani headed wide from just a few yards out.
The visitors were ahead just three minutes before the break when Feilhaber caused problems on the wing once again before picking out CJ Sapong, who had the simple task of heading home from close range.
Much to their frustration, Kansas were immediately put on the back foot, with Bedell and Tchani going close with efforts within the space of a couple of minutes.
The hosts grew in confidence after drawing level, but were unable to find a late winner as Federico Higuain curled the best of their opportunities wide from the edge of the box, before their numerical advantage was wiped off when Will Trapp was forced off the pitch due to a second bookable offence on Feilhaber.
Kansas took control in stoppage time, and the victory was secured in the third added minute when Feilhaber fired into the top corner from 25 yards out to leave Steve Clark helpless.