The result means that the Germans have moved one point ahead of the Gunners at the top of Group B after three games.
© PA Photos
Schalke were unlucky not be awarded a penalty with 14 minutes on the clock as Afellay went down under the challenge of Vito Mannone after a lack of communication in the Arsenal box. However, the striker was booked for diving by the referee.
The home crowd were on their feet midway through the half when Gervinho was put through on goal by Santi Cazorla, but there were soon groans of disappointment around the stadium as the forward's lack of pace saw him closed down by the visiting defence.
Meanwhile, the away side repeatedly threatened on the left as they looked to exploit Andres Santos.
The best chance of the half came in the final minutes of the opening 45 as Huntelaar somehow slipped the ball wide from five yards after Atsuto Uchida had eased past Santos.
Benedikt Howedes should have done better soon after the restart for the visitors, but he blazed the ball over the crossbar following good work from Jefferson Farfan.
It was Arsenal's turn to appeal for a penalty with 73 minutes gone, only for Gervinho to go into the referee's book for a dive over the boot of Jermaine Jones.
Schalke took a deserved lead soon after through Huntelaar's cool finish following the hosts' failure to clear their lines.
Vermaelen was unable to head the ball away as it pinged around the defence and Huntelaar was there to rifle the ball past Mannone and earn the three points for his side.