The Magpies had failed to record a win in their first seven matches of the season, which put manager Alan Pardew under severe pressure.
However, Gabriel Obertan scored the only goal of the game to fire the Tynesiders to the three points at St James' Park on Saturday.
Taylor told the club's official website: "There's relief all around the stadium. Obviously for the players it's what we've been working hard for, to get the first three points of the season.
"Now we can move on and hopefully it's a turning point for us."
Newcastle are 18th in the Premier League with seven points from their eight matches this season.