The hosts were beaten 3-1 by world number one side Australia in the bronze medal match at the Riverbank Arena, as their arch-rivals dominated the second half.
Lee has spent more than 25 years around British hockey - playing in Barcelona and Atlanta and coaching the men's team in Athens, Beijing and London - and he admitted his side were still suffering fatigue after their 9-2 defeat to Holland in the semi-finals.
"If you would have said seven years ago when we were awarded the Olympics that we might come fourth, we'd have taken that then," Lee said.
"Back in 2005 we were ranked 11th in the world and this result seemed a very long way away.
"Of course, we had aspirations and dreams of a medal but I believe Australia are still the best team in the world and it was daunting to face them after that defeat in the semi-final.
"However, finishing fourth in the Olympics is probably a fair reflection of where we stand in the world.
"I've given 25 years of my life to this and the last nine years coaching at the top level -
I've given my heart and soul to this team.
"There are tears of joy and an ocean of heartache in this job but I'm low on energy, my kids are not thankful for me being away and my wife isn't particularly tolerant.
"It's been a special time and great memories but maybe it's time for a rest."
Captain Barry Middleton was proud his side showed their character after their semi-final hammering but admitted the Olympics experience had left him 'flat'.
"It's been tough and there's been a lot of pressure because it's our home Games," he said.
"We've progressed up the rankings and got ourselves into a place where we are fighting for medals brings added pressure.
"No-one should make any emotional decision in the next few weeks but people need a break after putting so much into it.
"Hopefully we can pick up things again after Christmas and start looking towards Rio. We've come from 11th in the world in quick time but we want to go further, this is not the end of GB Hockey.
"The top three in the world are hard to chase but we want to chase them down."