The benchmark Nikkei 225 index ended down 0.45 percent, or 88.72 points, to 19,550.09 after spending most of the day in the red, as investors locked in profits following a sharp rally on Tuesday.
"Since the Nikkei average surged yesterday, profit-taking dominated the market mood in early trade," Okasan Online Securities said in a commentary.
The broader Topix index managed to add 0.04 percent, or 0.56 points, to 1,434.07.
An improving situation in some coronavirus hotspots has heartened traders, but Japan still faces uncertainty with a recent spike in Tokyo infections causing jitters.
"Japan still faces an expansionary phase of the viral infections. This is making investors cautious about Japanese shares," SMBC Nikko Securities said in a note.
Players were heartened by the news that Prime Minister Shinzo Abe was considering cash handouts to all Japanese citizens, allowing the Nikkei average to briefly shoot into positive territory, the brokerage added.
Investors held off from major moves to monitor the situation surrounding the coronavirus, including its impact on corporate earnings.
But selling pressure accelerated as investors dumped the dollar for the yen.
The US currency stood at 107.01 yen in Asian trade, against 107.21 in New York and 107.65 in Tokyo on Tuesday.
Among blue chip equities, Sony added 2.28 percent to 6,767 yen, while Nintendo gained 2.75 percent to 46,730 yen.
Construction equipment maker Komatsu rose 1.00 percent to 1,925.5.
Toyota lost 0.18 percent to 6,738 yen and fellow carmaker Honda was 0.82 percent lower at 2,371 yen.
Uniqlo operator Fast Retailing fell 0.72 percent to 48,440 yen, while SoftBank Group lost 1.97 percent to 4,333.