Oldham pushed for an opener early on and they nearly found it through Adam Rooney, whose long-range effort flew just wide of the post in the third minute.
The home side had a good chance minutes later as Danny Haynes got on the end of a cross, but the striker's header flew just wide.
Notts County took the lead in the 20th minute, however, as Haynes converted Greg Tempest's cross with a neat header from the centre of the box.
County pushed to take their lead further as Jack Grealish saw his headed effort creep wide. However Oldham grabbed an equaliser through Rooney right on the stroke of half time as the sides went into the break at 1-1.
The home side got off to a great start after half time when Yoann Arquin put County ahead in the 60th minute with a neat header, before McGregor unleashed a left-footed effort into the bottom corner as his side found themselves 3-1 up.
Oldham had not given up as they responded through Johnson Clarke-Harris's free kick, which found the bottom corner.