As the late afternoon shadows lengthened on the Old White TPC Course, Potter sealed the win with a four-foot birdie putt at the third extra hole, the par-three 18th, after Kelly missed his attempt from 45 feet.
Left-hander Potter had a golden opportunity to wrap up victory one hole earlier, at the par-five 17th, but he lipped out with a birdie putt from five feet after Kelly had rolled in a 20-footer to save par.
The two journeymen, who have played most of their golf on the 'mini' tours, finished the regulation 72 holes on 16-under-par 264, Kelly closing with a four-under 66 and Potter storming home birdie-par-eagle-birdie for a 64.
"This was just a big relief after all the struggles the last few weeks, knowing that now I've got a couple of years to try to improve on my game and win some more tournaments," Potter said after becoming the sixth first-time winner on the 2012 US Tour.
The 28-year-old, who earned his US Tour card by finishing second on the 2011 Nationwide Tour money list, had missed the cut in his five previous events.
"When you're missing cuts every week, you get down on yourself," said Potter, ranked 218th in the world. "But the plus side for me is I was still young. I just knew I had plenty of time, just be patient and it will come back around again."
South Korean Charlie Wi eagled the 17th on the way to a 65 to share third place at 14 under with US Tour rookie Charlie Beljan (67).
US Open champion Webb Simpson, two strokes ahead overnight in pursuit of a fourth US Tour title, tumbled backwards with three consecutive bogeys from the 12th to finish with a 73 in a five-way tie for seventh.
"It was just one of those unfortunate nine holes where all week everything had been going right and in nine holes everything couldn't have gone worse," Simpson said of his nightmare closing stretch.
For much of Sunday afternoon, the tournament appeared to come down to a head-to-head between world number five Simpson and 464th-ranked Kelly as they dueled for the title in the final pairing.
Kelly made a fast start, birdies at the second and fourth, where he hit a superb approach to three feet, putting him in a tie for the lead with Simpson at 14 under.
He then birdied the par-four fifth to move one stroke clear of his playing partner, who safely parred the first five holes.
Simpson regained a share of the lead when he sank a four-footer to birdie the seventh before Kelly fell back with a bogey on nine after finding the left rough off the tee and missing the green with his approach.
Kelly again moved into a tie at the top with a birdie at the 11th, where he knocked in a three-footer, before Simpson began his slide backwards with bogeys at the 12th, where he three-putted, and at the 13th.
Kelly briefly moved two ahead of the pack after sinking an 11-footer to birdie the 14th before Potter took the tournament into a playoff with his barnstorming finish.
While Kelly was bitterly disappointed to miss out on a maiden US Tour title, he was able to celebrate a spot at this month's Open via a six-tournament money list that ended on Sunday.
"I had a lot of fun, and I'm looking forward to getting in this kind of position some more," said the 33-year-old, who earned a late British Open berth for Royal Lytham along with Potter and Australia's Marc Leishman.
"When you're coming down the stretch, you've got to look back on things that you've had success in and just believe in yourself to hit those shots again," added Kelly, who turned professional in 2003.