Chris Brown, Demetrius Pinder, Michael Mathieu and Ramon Miller sped home in a Bahamian record time of 2 minutes 56.72 seconds, Miller passing American Angelo Taylor to huge roars from the crowd with just 50 metres left.
"Miller had a phenomenal leg. I really feel bad for these guys (team mates). I really didn't hold up the tradition," Taylor, who had been handed a lead of a couple of metres going into the last leg, said.
"I felt pretty good for 300 (metres), then I tried to hit it and I just didn't have it."
Bryshon Nellum, Joshua Mance, Tony McQuay and Taylor claimed silver for the US with a season's best time of 2:57.05, ending a run of seven successive golds in the event.
It was the Bahamas' first gold in men's track and field.
"No matter how small an axe, you can always bring down a big tree. We are a little axe and America is a giant tree, but we have done it," the Bahamas' Brown said.
Trinidad and Tobago's Lalonde Gordon, Jarrin Solomon, Ade Alleyne-Forte and Deon Lendore won bronze with hosts Britain just outside the medals in fourth after poor changeovers left them with too much to do.
South Africa, who had double-amputee Oscar Pistorius running the fourth leg of the race, finished eighth having got to the final on appeal after a collision caused by Kenya's Vincent Mumo Kiilu ended their qualifying heat.
Kenya's disqualification also robbed the final of the planned appearance of David Rudisha, who sprinted to 800 metres gold in a new world record time of one minute 40.91 on Thursday.
The last time the US lost on the track in the Olympic event was in Helsinki, 1952 where they finished behind Jamaica.
They did not win in 1972 because they withdrew from the competition, or in 1980 when they boycotted the Moscow Games.
The US took gold in 2000 in Sydney but victory was eventually awarded to Nigeria after one of America's sprinters in the race Antonio Pettigrew admitted to doping offences.
The US had beaten the Bahamas in Beijing in 2008 to take gold in an Olympic record time of 2 minutes 55.39 seconds - the second fastest performance ever - but were much weaker in London and had already been edged out by the Bahamas in the heats.
American 2008 Olympic 400m champion LaShawn Merritt, a first-choice relay option, injured his hamstring moments into his title defence on Saturday and pulled out of the Games, while Manteo Mitchell suffered a broken leg during the relay heats.
Double US relay gold medallist Jeremy Wariner was also unfit to run in London.
The US still remained favourites for gold and their chances looked to have been bolstered on Thursday when medal contenders Jamaica crashed out in qualifying after Jermaine Gonzales pulled up injured and ended their race.
The world's fastest man Usain Bolt had said there was a slim chance he might have run the final if Jamaica had got there.
Frenchman Renaud Lavillenie produced an Olympic record jump of 5.97 metres to win gold in the men's pole vault as silver and bronze went to Germany.
Germans Bjorn Otto, 34, and his 22-year-old compatriot Raphael Holzdeppe cleared 5.91m, but Holzdeppe had more missed attempts which meant Otto took silver.
European champion Lavillenie, who also holds the world and European indoor championship titles, had been favourite for gold and looked the part in his first three vaults, sailing over at the first attempt each time to roars from the crowd.
The race for gold quickly whittled down to Lavillenie versus the German duo, who took the momentum with first-time clearances at 5.91 metres - a new personal best for Holzdeppe - while Lavillenie failed and then passed at the mark.
But the glory was to go to the Frenchman who took bronze at the world championships in Daegu last year, sailing over at 5.97 without reply from his German rivals.
"I can't really believe it. Maybe tomorrow I will wake up and realise that I am an Olympic champion. The Germans were very strong and they pushed me to my limits. This has to be the best thing in my life," the 25-year-old Lavillenie said.
Otto is in the best form of his career this year, finishing second to Lavillenie at the world indoor championships and European championships, but said he would not rest with second.
"He (Lavillenie) has won every gold this year and beating me on many occasion, so he must be very proud. But the year is not over. There is still a bill that needs to be settled," he said.
Britain's Steve Lewis was fifth with an impressive 5.75m, while Australia's defending champion Steve Hooker, who has suffered from a crisis of confidence this season, made a swift exit after failing to clear his first height of 5.65 metres.
The 30-year-old had passed on the first mark of 5.50 but looked nervous before all three attempts at 5.65, crashing down onto the mat with the bar in his hands on his final try.
He then had to watch on as Lavillenie beat his Olympic record of 5.96m set in 2008.
After winning in Beijing, Hooker's dominance extended to winning the 2009 world championship and the indoor title in 2010, but he has suffered a miserable time since, first with injury and then with his confidence.
"My technique was off a bit on the first jump. It kind of threw off my rhythm, I'm not quite getting things right on the runway," Hooker said.