The Selecao looked to be heading into the semi-finals in the 90 minutes after Robinho's early goal.
However, a 72nd-minute penalty from Derlis Gonzalez saw the Paraguayans force a shootout before misses from Brazil substitute Everton Ribeiro and another from Douglas Costa meant that Dunga's side crash out of the tournament.
The Brazilian team had made a strong start with Philippe Coutinho firing over the bar inside the opening two minutes before the Selecao took the lead with only a quarter of an hour played in Concepcion.
Their breakthrough came after a pinpoint cross from Dani Alves, which allowed Robinho to tap in from close range.
Just after the hour mark, Paraguay again threatened an equaliser as Paulo Da Silva's header was beaten away by Jefferson, but Los Guaranies were awarded a penalty with 18 minutes remaining after Thiago Silva's handball.
Gonzalez duly stepped up to give Paraguay a route back into the match and they were able to force a penalty shootout.
Brazil's luck worsened as first Ribeiro and then Costa missed from the spot to gift the Paraguayans a place in the last four.