Forest, previously unbeaten in four trips to this part of South London, remain six points off a place in the top half of the division.
The home side were on their way to a 14th match without defeat at The Den moments after kickoff as Williams found a way through with a close-range header.
Boro were desperately struggling to get any sort of foothold on the game as the first half transpired, with Millwall attacking at will and coming close to doubling their tally.
The second goal would arrive 10 minutes before the interval through Gregory, who was teed up nicely by Steve Morison to drill low into the bottom corner.
Millwall could have had a third thanks to Jake Cooper's hooked attempt, which was cleared off the line by a covering opposition player.
Gregory then felt that he should have had a late penalty under a challenge from Costel Pantilimon, though the referee's reluctance to point to the spot mattered little as the Lions saw things through to extend their unbeaten run.
MILLWALL (4-4-2): Archer; Romeo, Cooper, Hutchinson, Meredith; Wallace (Onyedinma 88'), Saville, Williams, Marshall (O'Brien 89'); Morison (Shackell 91'), Gregory
NOTTINGHAM FOREST (4-2-3-1): Pantilimon; Dariwka (Worrall 67'), Figueiredo, Fox, Osborne; Watson, Colback; Lolley (Brereton 67'), Tomlin, McKay (Cash 55'); Vellios