Substitute Semedo, on loan from Benfica, struck his first Forest goal five minutes from time to extend his side's unbeaten run to eight games with a 1-0 victory.
It was no less than Forest deserved against a Swansea side, who stay top of the Sky Bet Championship but would have secured their best ever start to a season with a sixth win from seven games.
But Forest were the better side for large parts of the first half and sliced through the Swansea defence after 16 minutes.
Forest went even closer when Ameobi's cross was only cleared as far as Ben Watson 20 yards out.
The midfielder headed the ball back into the box and the stretching Grabban turned it on to the Swansea crossbar.
The Forest goal was not threatened on either occasion, but Michael Dawson reacted smartly to block Borja's effort at the near post two minutes before half-time.
Forest remained bright after the interval and Grabban latched on to Bersant Celina's back pass to set up Carvalho for a chance which Woodman kicked away at his near post.
Celina was soon substituted after that lapse and Swansea gained some overdue momentum when Andre Ayew headed Bidwell's cross narrowly wide.
But Woodman was again to the rescue to tip Joe Worrall's volley around a post and Joe Rodon, back from international duty after making his debut for Wales, blocked Sow's effort from the resulting corner.
Ayew was becoming an increasing influence on the contest and he tested Samba with a low drive before winning the free-kick which saw the Forest goalkeeper punch clear under pressure.
Forest continued to stand firm and Grabban's strong running and unselfish attitude eventually sealed the points.