It is the third successive trip to the Olympic semi-finals for Argentina, who won gold at the 2004 Games and bronze in Beijing, and they hopped up and down in joy as a group and then sang along with their fans in celebration.
"We hung in there in the end and got an awesome win," said Argentine veteran Manu Ginobili. "It's very, very hard to make an Olympics semi-final and against Brazil for us it's maybe even more special."
Argentina will likely face the defending champions United States, a prohibitive favourite over Australia, who were meeting in the final quarter-final of the day.
Spain and Russia set up a semi-finals showdown in the other bracket on Friday.
Russia beat Lithuania 83-74, and 2008 silver medallists Spain defeated France 66-59 in their quarter-finals.
Luis Scola led a balanced Argentine attack with 17 points. Ginobili and Carlos Delfino contributed 16 points each.
Marcelinho Huertas kept Brazil rolling in the first half with a hot shooting hand as he made 7-of-11 shots, including three three-pointers, for 17 points as they led 26-23 after the first quarter and trailed 46-40 at the break.
Leandro Barbosa had only two points in the first half, but the Brazilian ignited a comeback from 14 points down with 20 second-half points that took them within two points at 70-68 before the Argentines pulled away in the last two minutes.
Huertas and Barbosa each had 22 points for Brazil.
"It was an unexpected game," said Ginobili.
"I didn't expect us to score 46 points in the first half. It was an offensive-minded game. In the second half it got a lot tighter."
Scola said he was not surprised that Brazil battled their way back.
"They came really close. We knew they were going to come back. It pretty much always happens," said Scola.
"I dream about the podium, it's one game away."