The visitors should have taken the lead in the second minute when Eric Alexander raced onto a loose ball and was one-on-one with Steve Clark, but the keeper managed to spread himself to make the block.
New York went close again when a long ball over the top was chased down by Bradley Wright-Phillips, but he could only put his left-footed shot wide of the post.
The Crew responded and Waylon Francis forced Luis Robles into making a great save as his left-footed shot looked to be finding the net, but the keeper's fingertips kept the shot out.
After half time, Thierry Henry almost brought his side level as he volleyed an effort from the edge of the area only to see his shot well saved by Clark.
Columbus were within inches of doubling their lead as the ball came to Arrieta following an excellent team move, but his left-footed shot hit the post and he put the rebound over the bar.
Arrieta was to rue this missed chance as the Red Bulls drew level when Wright-Phillips controlled a cross from Lloyd Sam and then placed the ball into the bottom corner with the aid of the post.
Both sides pressed for a winner but were forced to settle for a point each.