The Hammers had taken the lead through Mark Noble's spot kick just 19 minutes into the game at the bet365 Stadium but could have been three to the good by the break, with Arnautovic first losing out in a one on one with Jack Butland and then sending another shot over the bar.
Moyes's side did eventually make the result certain as the contest came to a close, with Arnautovic atoning with 15 minutes remaining and Diafra Sakho adding late gloss.
The result extended West Ham's unbeaten run to three games and helped move them out of the relegation zone for the first time since Moyes took over.
"The result went our way and it was important win. We're gaining momentum and at times we showed that," he told Match of the Day afterwards.
"We had to deal with a lot of stuff but we missed good chances. I was annoyed with Marko at half time as he had the chance to get two. He's doing really well and I said to him at half time he would score.
"Clean sheets are one of the things you want to do. Don't shout about it too much as we're going to need a few more."
Next up for Moyes's side is a home encounter with struggling Newcastle United on Saturday afternoon.