Taibu played 28 Tests, 150 one-day and 17 Twenty20 internationals from 2001 to February this year, since when a finger injury had sidelined him.
The decision is surprising not only because of Taibu's age, but also because earlier on Tuesday he was named in Zimbabwe's provisional squad for the World Twenty20.
Taibu, a Christian, stated he wants to focus on working for the church, and he said: "I just feel that my true calling now lies in doing the Lord's work and, although I am fortunate and proud to have played for my country, the time has come for me to put my entire focus on that part of my life."
Taibu was an automatic pick for Zimbabwe since making his international debut aged 18, except for the times when he clashed with his country's cricket board. He had stepped down as skipper and quit the national side in 2005 following threats against his family.
Taibu moved to South Africa in 2006 with the intention of going through the four-year qualification process to be eligible for international cricket for them. However, he reappeared for Zimbabwe in 2007.
One of the highlights of his career was a man-of-the-match performance in 2005 against Bangladesh when he made 85 not out and 153 to help Zimbabwe draw the Test.
His only Test victory was against the Tigers last year when Zimbabwe returned to the format after a six-year exile.