Tries from Sona Taumalolo and Liam Messam helped the Chiefs race to a 17-6 lead as the 40-minute siren sounded at Waikato Stadium but Ryan Crotty touched down from the final move of the first half and Dan Carter kicked the visitors back to within three points.
Carter looked set to further stoke the Crusaders' chances of
adding to their record seven titles when he lined up a penalty in front of the posts with four minutes remaining.
The kick would have tied the semi-final at 20-20 but it dropped just short and the Chiefs held on until the whistle in the face of constant pressure.
"It's pretty unreal at the moment. I just want to look at the scoreboard and let it sink in a little bit," Chiefs captain Craig Clarke said.
"Their try before half-time certainly made the game interesting. We said we were going to have to come out for the second half and be strong. It certainly wasn't perfect but it was good enough."
There will now be a new name on the trophy, with the Chiefs set to meet the Stormers or the Sharks in next week's final.
The South African rivals meet in Cape Town in Saturday's second semi-final.
The Chiefs lost their only previous final to the Bulls in 2009, while the Sharks have lost three finals and the Stormers one.
Three weeks after losing to the Crusaders at the same venue, the Chiefs' searing pace and high defensive line forced a series of uncharacteristic errors from their rivals.
"We made crucial mistakes and never really got our game going," Crusaders captain Richie McCaw said. "They've set the standard over here, there's no doubt about that. They've played well all the way through."
The score was 3-3 following an exchange of penalties between Aaron Cruden and Carter when Sonny Bill Williams broke a couple of tackles and offloaded in midfield.
Quick hands from Cruden and Tawera Kerr-Barlow gave lock Brodie Retallick the chance to carry to within striking range of the line, from where Taumalolo picked up and drove low.
The Tonga prop appeared to propel himself a second time to reach the line but referee Craig Joubert referred the decision to the TV official, who ruled that the score was good.
There was no ambiguity about the next try.
Fullback Robbie Robinson broke from deep and cut through tackles with ease before being hauled down. Williams picked up and offloaded to Messam, who danced through to score and Cruden added the extras for 17-6, with Carter having struck a second successful penalty for his side.
A tight penalty call during a chase downfield then stopped the Chiefs earning at least a five-meter scrum. The decision allowed the Crusaders to return play to the other end and last year's beaten finalists made the most of their reprieve.
Matt Todd broke through a gap, Carter kept the ball moving and Crotty sneaked in to trim the deficit to 17-11 at the interval.
From the restart, Carter almost immediately added a third penalty to his tally for 17-14 and traded kicks with Cruden to make it 20-17.
With McCaw marshalling his forwards, the Crusaders pummelled the Chiefs in the hope of a breakthrough but the closest the teams came to adding more points was when Cruden hit the underside of bar with a penalty from 55 meters and Carter failed with his late attempt.
That missed kick means the Chiefs are only the second New Zealand side to beat the Crusaders in 10 playoff matches. The Crusaders' only previous defeat to another Kiwi outfit was against the Auckland Blues in the 2003 final.