The 24-year-old, a £6m summer signing from AS Monaco, scored twice to help Newcastle reach the fourth round of the League Cup with a 3-2 win at Selhurst Park.
Having previously gone six games without finding the net, Riviere believes that Wednesday's performance will prove a watershed moment in his spell in the North-East.
"I'm happy to get my first two goals. I didn't worry about not scoring because in training I have been playing well but the goals just weren't coming," the Frenchman told reporters.
"But I kept believing in myself. I know what I can do. I was waiting for that first goal to come and, now it is here, I know that more will follow.
"I want to play the next game and to keep going like this."
Alan Pardew's side resume Premier League commitments on Monday night with a trip to Stoke City.