Rotherham, with just one win in their previous nine games, made six changes of their own but looked far more cohesive and they deserved more than just Matt Olosunde's maiden senior goal for their efforts.
Cenk Tosun's first strike for Everton since November 2019 was cancelled out just after half-time by the former Manchester United academy graduate, who won a solitary cap for the United States in 2018.
Substitute Abdoulaye Doucoure scored the eventual winner three minutes into the additional period but it was far from comfortable for Ancelotti's side, who progressed to the third round for only the second time in five seasons.
What began with the appearance of a behind-closed-doors friendly arranged primarily for James Rodriguez to sharpen his fitness, given free rein by Ancelotti to roam where he liked, turned into something of an ordeal for Everton.
The Colombia international's movement pulled Rotherham's defence out of position to allow the 19-year-old Anthony Gordon to turn his marker Michael Ihiekwe and his inviting through-ball down the inside-left channel was dinked over the advancing Jamal Blackman by Tosun in the ninth minute.
This week the Turkey international, with one eye on this summer's Euro 2020, said he was considering his future this month and while the goal was a much-needed confidence boost little else about his play suggested he has much to offer Ancelotti.
His goal should have given the hosts a platform but in the 30 minutes to half-time Rotherham were the better side with Robin Olsen making a double save from Olosunde and then Dan Barlaser before Matt Crooks flicked a near-post header onto the roof of the net.
Despite having the lead Everton were second-best for the remainder of the first half and within 10 minutes of the second were made to pay.
Wing-back Olusunde capitalised on a weak Tom Davies header back towards his own goal to turn and fire a shot across Olsen and inside the far post.
Ancelotti sent on Doucoure, Bernard, Yerry Mina – replacing Lucas Digne after his first appearance since ankle surgery in November – and Gylfi Sigurdsson in two quick double substitutions in an attempt to change the momentum.
Everton won their first corner in the 73rd minute from a Rodriguez deflected shot but it was hardly the turnaround the Toffees boss expected.
Tosun thought he had won it three minutes from time by diverting in Rodriguez's free-kick, but the goal was ruled out by VAR for offside.
But three minutes into the additional period there was no denying Doucoure, who slotted home from Rodriguez's pass to make it 2-1.
Bernard should have killed off the game but hit the post but the visitors could not force another equaliser.