Cox's spot kick was saved by Wayne Hennessey and the Welsh international managed to block the follow up.
Forest were made to pay when Upson doubled Yeovil's advantage five minutes befor the break with a goal almost a carbon copy of his first beating Darlow from similar range.
Chelsea loanee Nathaniel Chalobah managed to pull one back when he got on the end of Jack Hobbs's header and nodded home from close range.
But Yeovil mustered a third just before the half-time whistle, which was the prove crucial as Byron Webster's headed trickled across the line although the visitors' players disagreed.
Forest pressed for a way back into the game, registering 27 shots on goal in the course of the game and having nearly two thirds possession.
But they were unable to breach Yeovil's goal and the result sees the Somerset outfit leapfrog Bolton and Sheffield Wednesday in the Championship.
Despite defeat, Forest remain in the playoff places, albeit nine points behind leaders Burnley.