Sammer, who was released from his position as sports director at the German football association (DFB) earlier on Monday to accept Bayern's offer, will start work immediately as the club seeks to end Borussia Dortmund's two-year reign in Germany.
"I wish Bayern all the best as I enjoyed a very intense and good time here," said Nerlinger, who had replaced current club president Uli Hoeness in 2009.
Bayern finished runners-up this season to Sammer's old club Dortmund in the Bundesliga and German Cup while also losing to Chelsea in the Champions League final.
The 1996 European Footballer of the Year, who was part of Germany's last title-winning team at Euro 96 and also won the 1997 Champions League title with Dortmund, joined the DFB in 2006 and had a contract to 2016.
Sammer also enjoyed success as a coach, leading Dortmund to the 2002 Bundesliga title and at 34 became the youngest league-winning coach, a record he still holds.
"We let Matthias Sammer go with a heavy heart," said DFB president Wolfgang Niersbach. "The close relationship with him will remain."