Ireland were indebted to substitute Kevin Doyle for volleying a superb injury-time winner after a Robbie Keane penalty in the 89th minute had cancelled out Kairat Nurdauletov’s 37th-minute header.
Kazakhstan, ranked 116 places below Ireland, will be mortified after allowing a famous win to slip from their grasp.
Kazakhstan, ranked 142 in the world, started with just one player who earned a living outside the country: Heinrich Schmidtgal plays for SpVgg Greuther Furth, who won promotion to the Bundesliga last season.
McCarthy had a great chance in the second minute with a low and finely judged side-foot shot that brought a good save from Kazakhstan goalkeeper Andrey Sidelnikov.
However as time went on they began to look for the direct route almost exclusively, with defenders Sean St Ledger and Stephen Ward constantly trying to find the head of Walters.
It was one-dimensional and allowed Kazakhstan to grow into their own game and take the initiative.
Sergey Ostapenko and Tanat Nusrbayev had half-chances before the home side took the lead.
Ward gave away a cheap free-kick on the right when he tripped Ulan Konysbayev instead of holding the midfielder up on the touchline.
Schmidtgal's delivery was superb and Ordabasy midfielder Nurdauletov flicked a header over Ireland goalkeeper Keiren Westwood and into the far corner of the net. However, Ireland manager Trapattoni will question the marking, with Glenn Whelan allowing the Kazakhstan captain far too much room.
Ireland in the second half still looked keen to play the ball long, and had only their second shot on target in the 72nd minute when Whelan's speculative drive arrowed straight at Kazakhstan goalkeeper Andrey Sidelnikov.
Kazakhstan substitute Baurzhan Dzholchiyev had a great chance to make it 2-0 in the 78th minute, but Westwood did brilliantly to save with his legs.
Doyle then took control of the game. Picking up the ball on the halfway line, he drove to the left touchline and crossed towards Keane. Mukhtar Mukhtarov deflected the ball towards goal, and Mikhail Rozhkov pulled back Keane when the Ireland captain tried to poke home the loose ball.
Keane slammed home the resulting penalty - and seconds later Ireland hit an unlikely winner.
Whelan dinked a ball into the box and Ward headed the ball towards the penalty spot, giving Doyle the chance to slam a left-footed volley past Sidelnikov.
Ireland's next Group C fixture is against Germany on October 12 at the Aviva Stadium in Dublin.