As a 16-year-old Guor Marial moved to the United States, where he has permanent resident status.
Given South Sudan, the world's newest country recognised only last year, has not yet established a national Olympic Committee - and so cannot send a team to the Games starting next week - Marial was unable to represent that country.
As he is not a US citizen, he was also ineligible to take part for the Americans.
The IOC had initially suggested Marial run for Sudan, which has invited him to join their team, the runner told AlertNet, a humanitarian news service run by Thomson Reuters Foundation.
Marial, who lost 28 members of his family in the civil war, refused.
Sudan and South Sudan came close to all-out war in April following border clashes, the worst violence since South Sudan seceded and declared its independence from Khartoum a year ago under a 2005 peace agreement that ended decades of civil war.
The 28-year-old reached the Olympic qualification time in October last year and improved his personal best in San Diego, California, last month, finishing in two hours 12 minutes 55 seconds.
Three athletes from the former Netherland Antilles will also compete under the Olympic flag.