The defender popped up in the box in the 88th minute and struck the ball past opposition goalkeeper David Forde.
Speaking to talkSPORT, Upson said: "It's a great feeling. I guess it's one of those things in our first appearance in the Championship. It was great to get a last minute winner.
"I think someone asked the question about how we will cope with [the Championship], but the pressure is off us. We just need to play our game."
Yeovil's next fixture in the Championship is at home to Birmingham City next Saturday.