Nadal, 26, cited physical problems following a gruelling season, saying his withdrawal would allow a better-prepared Spanish player to take his place.
He had been set to be a flag-bearer for Spain at the opening ceremony on July 27.
"I am not in the right physical condition to compete at the London Olympics and for this reason I will not travel as arranged with the Spanish delegation," the world number three announced.
"I have to think about my team-mates, I cannot be an egotist and think about myself here. I have to think about Spanish sport, and that one of my team-mates who is better prepared than me can take my place and compete to the best of his ability."
Nadal, who won his 11th Grand Slam title and seventh French Open earlier this season, is a former world number one and widely regarded as one of the best players of all time.
However, his all-action style of play - and tendency to feature in epic five-set matches - has taken its toll and injury problems have blighted his recent career.
The Mallorca native was widely expected to be a medal contender at the Games, which are being held at Wimbledon, where he exited to unheralded Czech Lukas Rosol in the second round of the All England Championships in June.
"I've waited until the final moment of my preparation and my training, but I cannot do it," he added.
"Today is one of the saddest days in my career. Carrying the flag for Spain at the opening ceremony was set to be one of the biggest honours, one of the most special moments for me.
"For this reason I hope you can understand how difficult it has been for me to take this decision."
The favourites for men's gold are Roger Federer, who has never won the Olympics but claimed this year's Wimbledon title to move back to world number one, world number two Novak Djokovic and British hope Andy Murray.