The benchmark Nikkei 225 index fell 0.49 percent or 107.22 points at 21,687.10 while the broader Topix index was down 0.38 percent or 6.64 points at 1,725.30.
Anxious investors rallied after a conciliatory speech by Chinese President Xi Jinping reassured them a trade war with the United States could be avoided.
"Profit taking emerged as (the Nikkei) neared 22,000," said Okasan Online chief strategist Yoshihiro Ito.
But, "a wait-and-see mood grew due to concerns about (Japan's) domestic politics and geopolitical risks arising from Syria," he said in a commentary.
Japan's Prime Minister Shinzo Abe was being grilled in parliament Wednesday over cronyism allegations that have sent his cabinet's public support ratings plunging in recent weeks.
Internationally, tensions remain high after UN members failed to agree on action to probe alleged chemical attacks in war-wracked Syria.
Washington has maintained the threat of missile strikes in response to the attacks, despite the risk of triggering a head-on clash with Damascus ally Russia.
The dollar was at 107.07 yen against 107.19 yen in New York Tuesday afternoon.
SoftBank Group closed up 3.44 percent at 7,980 yen after the Wall Street Journal reported that US carriers in the group -- Sprint and T-Mobile -- have rekindled merger talks.
Toyota edged up 0.10 percent to 6,845 yen and shipping firm Nippon Yusen climbed 1.78 percent to 2,164 yen.