The 25-year-old spent the 2013-14 season on loan at the Riverside, making 34 league appearances and ending the campaign as the club's first-choice right-back.
Boro were reportedly keen on signing Varga permanently, but he believes that the North-East club can't afford to prise him away from parent club Debrecen.
"It seems their budget doesn't allow them to buy me permanently," said Varga at a press conference ahead of Hungary's friendly with Denmark on Thursday.
"This is not a matter of my football knowledge and it's not a personal question [that they didn't buy me] but [the problem is that] they couldn't reach their goals - they haven't been able to win promotion for years so they can't really afford to buy me."
Boro have been in the Championship for five seasons since their relegation from the Premier League in 2009. They finished the 2013-14 season in 12th place, eight points shy of the playoffs.