The five-time Flushing Meadows champion dropped the first set for the second successive match but overcame world 99 Damir Dzumhur in four.
Federer, who lost the opening set to Indian qualifier Sumit Nagal in the first round, was broken in his first service game thanks to a wayward volley and a loose forehand into the net.
Dzumhur, of Bosnia, then secured an unlikely double break to take a 4-0 lead on his way to clinching the first set.
The drama almost turned into a crisis as Federer faced a break point in the second, but he survived a long, tense rally to come out unscathed.
From then on the 20-time grand slam champion took control, levelling the match courtesy of a couple of Dzumhur double faults.
Dzumhur gave Federer few more problems and the 38-year-old wrapped up a 3-6 6-2 6-3 6-4 win in two hours and 22 minutes.
"I buckled down and told myself to stay tough and hang with him," said Federer afterwards. "That will help my confidence for the next match."
Earlier seventh seed Kei Nishikori beat American Bradley Klahn.
The 29-year-old from Japan, a finalist in New York in 2014, won 6-2 4-6 6-3 7-5.