Astrit Ajdarevic had the first chance of the afternoon, but he saw his long-range effort deflect wide of the post.
Jordan Cousins was next to threaten for Charlton, but after a driving run through the midfield, his scuffed shot was easily saved by the keeper.
After the break Adam Le Fondre wasted a big chance for the visitors when he burst into the box and fired wide of the post.
In the 73rd minute Reading made their pressure count, as Williams found the back of the net with a powerful strike into the top corner.
Reza Ghoochannejhad scuffed an effort wide for Charlton in the closing stages as the visitors held on for the three points.