Song, ranked 17th in the world, had not been expected to challenge for gold but he upset Cuban Asley Gonzalez in the final of the men's -90kg category, his winning waza-ari throw coming in the golden score extra time period.
The 33-year-old Korean said it had taken more than 20 years to achieve his dream of winning Olympic gold when he took up the sport as a 12-year-old.
"I was about to retire four years ago but I pushed back that decision," said Song, who added he would now quit the sport to coach young judokas.
Despite his age and lack of previous major competition titles, Song said he had never lost hope.
"When you become a winner it's important not to become complacent and when you become a loser it's important not be disappointed or frustrated."
Asked about being an outsider for the title, he said: "I think all the athletes competing in the Olympic Games are equally capable of winning a gold medal and thanks to a lot of preparations I believe I was able to win."
His title is the second judo gold medal for the South Koreans at the London Games.
Japan's Masashi Nishiyama took bronze with a win over Russia's Kirill Denisov, who had earlier put out Greek favourite Ilias Iliadis, but it still leaves the Japanese men chasing their first judo gold in London.
Iliadis, a gold medallist in Athens in 2004, beat Brazil's experienced Tiago Camilo for the other bronze.