The deadlock was broken midway through the first half when poor marking from the hosts allowed Grant Ward, who secured a loan move to the Millers from Tottenham Hotspur earlier this week, converted a cross from close range.
However, the Championship side were left stunned as Alfreton hit back before the break through Andrai Jones, who picked out the top corner with a powerful strike from 25 yards.
Rotherham were given a scare early in the second half when John Johnston found the space to rattle the post with a low strike before Adam Collin reacted well to keep out the headed rebound from Dan Bradley.
The visitors survived the spell of pressure to regain the lead just after the hour mark when Ward found the space to score his second of the night from a tight angle in the box.
Alfreton continued to push for a way back into the match, and they were rewarded with an equaliser 11 minutes from time as Johnston latched on to a cutback in the cross to slot home calmly.
There was still time for a late twist on an entertaining evening as Rotherham were punished for failing to clear their lines when Joe Ironside headed home from a few yards out.