Airport passport desks around London have struggled to cope in recent months and some passengers have had to queue for several hours to get through immigration checks at Heathrow Airport because of the shortage of staff.
The Public and Commercial Services union (PCS) - one of Britain's most militant - said its members, including border agency and immigration personnel, had backed strike action as part of a row with the interior ministry over job cuts and pay.
The Olympic Games begin on July 27, and hundreds of thousands of visitors are expected in London every day while the Games are under way, with Aug. 13 predicted to be the busiest day ever at Heathrow, Europe's busiest airport.
Union chiefs said they would announce on Thursday the dates and type of action they would take, but the government said the low 20 percent turnout in the ballot meant the union would have little public backing.
One PCS source said union leaders were actively considering strikes on some of the Games' busiest days of the Games. "We will be looking at key dates," the source told Reuters.
The union says the long delays at Heathrow, caused by a shortage of passport staff, are due to a 22 percent cut in the number of border staff, part of a government cost-cutting exercise to shrink the record budget deficit.
Some passengers arriving at the airport's Terminal 4 on April 30 had to queue for three hours to reach passport control.
"Ministers have known about these issues for a very long time but have chosen not to act," PCS general secretary Mark Serwotka said in a statement.
"We believe they have acted recklessly and irresponsibly in cutting so many jobs and, in the case of UKBA (UK Border Agency), they have simply tried to paper over the cracks by deploying severely undertrained staff at our borders."
The union said 20 percent of its members had taken part in the ballot, 57 percent of these backing a strike and 75 percent supporting other forms of industrial action.
The government, which has drafted in extra border staff to help cope with the Olympic rush, said these figures showed the union had no authority to call any strike.
"A decision to strike is completely unacceptable and we believe the public will have no sympathy with the union's decision," said Immigration Minister Damian Green.
"The security of the UK border is of the utmost importance and we will use our trained pool of contingency staff to ensure we minimise any disruption caused by planned union action.
"Any action that disrupts the Olympics will be completely unacceptable and the public will not support it."
The government has already come under fire after a private firm admitted last week it would not be able to train enough security guards in time for the Games, forcing the deployment of 3,500 troops to make up the shortfall.