Andrija Zivokic had a great chance midway through the first half after a cross from Filip Kostic, but the 19-year-old lifted his shot over the bar from close range.
However, Serbia went ahead minutes later after Kostic's ball found Zivkovic in the box again and his attempt hit Levon Hayrapetyan, who accidentally put the ball past his own goalkeeper.
Ljajic made it 2-0 just after half time as a quick counter-attack caught out Armenia and the new Inter Milan loanee finished from just outside the six-yard box.
Borussia Dortmund forward Henrikh Mkhitaryan tried to pull his side back into the game with two efforts in the final two minutes but Serbia stood firm to earn the three points.
Both sides will miss out on qualifying for the European Championships was they sit on just one point after Serbia were deducted three points for their conduct against Albania back in October.