The two nations have a long-standing dispute over an uninhabited group of islands in the East China Sea - known as the Senkaku in Japan and Diaoyu in China.
The row escalated on Tuesday after two Japanese activists landed on one of the islands with Beijing describing the move as provocative. China lodged a complaint with Tokyo and said it reserved the right to take further action.
The move came on the highly-charged anniversary of Japan's occupation of its giant neighbour in 1931.
The Japan Rugby Football Union has therefore decided against making the weekend trip to Shanghai, the Kyodo news agency reported.
"We've been training hard to try to win the competition so this is disappointing," said coach Tomohiro Segawa. "We just have to move on from this and keep working."
The Chinese have also withdrawn their badminton players from this week's Japan Open while a Japanese cycling team have been kicked out of a race in China.
In addition, Japan's Olympic silver medallist Kasumi Ishikawa was pulled out of a World Cup table tennis event because Chinese organisers could not guarantee her safety.
Japan's rugby players lead the way in the four-leg Asian Series following their win in the opening Borneo Sevens.
The penultimate leg will be held in Mumbai followed by the final event in Singapore which will double up as the Asian qualifier for the Rugby World Cup Sevens in Moscow in 2013.