Phelps anchored his team's race and clinched gold for the United States to seal his 15th gold, with two silvers and two bronze, surpassing Soviet gymnast Larisa Latynina as the most decorated Olympian of all time.
France took the silver in the relay, while China won the bronze.
The Americans were overwhelming favourites to win the relay and the result was never in doubt after Ryan Lochte gave them the lead after the lead-off leg.
Conor Dwyer and Ricky Berens kept America's lead by the time Phelps dived in for the anchor leg and he brought them home in a combined time of six minutes, 59.70 seconds.
Lochte captured his second gold medal in London after winning the 400 individual medley on the opening day of competition and the fifth of his career.
France, who upset the Americans in the 4x100 freestyle final on Saturday, took the silver medal while China grabbed the bronze after Sun Yang anchored them home.
Phelps record-tying second silver came in frustrating circumstances behind Chad Le Clos of South Africa, however, in a nail-biting finish to the men's 200m butterfly earlier in the evening.
The American swimmer led going into the last few metres of his favourite race, but was tiring rapidly and had to settle for second when Le Clos ploughed through to snatch it on the final touch by five hundredths of a second.
"I felt like him, swimming that last 50 I felt like I was Phelps," said Le Clos. "I always wanted to swim in an Olympic Games and I wanted to be like him.
"I always remembered Phelps coming off the last wall strongly, using that momentum, so that's what I tried to do."
Phelps, who sealed his legacy on Tuesday with a record 19th Olympic medal, praised Le Clos's performance, describing him as "a very, very good competitor, a very hard working kid, a very hungry kid".
Takeshi Matsuda of Japan claimed bronze, 0.2 seconds behind Phelps and over a second ahead of Austria's Dinko Jukic in fourth.
Watching at poolside was Latynina, who has held the record for decades with her haul - including nine golds - from the Games at Melbourne (1956), Rome (1960) and Tokyo (1964). Latynina admitted earlier this month she had no doubt Phelps would overtake her in London and "I can only wish him well".
MEN'S 200M BUTTERFLY RESULTS:
1. Chad Le Clos (South Africa) 1:52.96
2. Michael Phelps (US) 1:53.01
3. Takeshi Matsuda (Japan) 1:53.21
4. Dinko Jukic (Austria) 1:54.35
5. Tyler Clary (US) 1:55.06
6. Velimir Stjepanovic (Serbia) 1:55.07
7. Pawel Korzeniowski (Poland) 1:55.08
8. Chen Yin (China) 1:55.18
MEN'S 4X200M FREESTYLE RELAY RESULTS:
1. U.S. Ryan Lochte/Conor Dwyer/Ricky Berens/Michael Phelps 6:59.70
3. China Hao Yun/Li Yunqi/Jiang Haiqi/Sun Yang 7:06.30
4. Germany Paul Biedermann/Dimitri Colupaev/Tim Wallburger/Clemens Rapp 7:06.59
5. Australia Thomas Fraser-Holmes/Kenrick Monk/Ned McKendry/Ryan Napoleon 7:07.00
6. Britain Robbie Renwick/Ieuan Lloyd/Rob Bale/Ross Davenport 7:09.33
7. South Africa Darian Townsend/Sebastien Rousseau/Chad Le Clos/Jean Basson 7:09.65
8. Hungary Dominik Kozma/Laszlo Cseh/Peter Bernek/Gergo Kis 7:13.66