The Nikkei 225 lost 1.34 percent or 286.01 points to close at 20,031.31, while the broader Topix index fell 1.02 percent or 17.57 points to 1,699.56.
"The Tokyo market is affected by falls on Wall Street and the fact that today is an ex-dividend date," or the date buyers lose their rights to dividends, Kyoko Amemiya, senior market adviser at SBI Securities, told AFP.
Shares were also affected by bad news for the electric carmaker Tesla, including the US authorities' investigation into a fatal crash involving a Tesla vehicle last week in California, she said.
"The currency market is reacting calmly" to news of talks between North Korean leader Kim Jong Un and China's President Xi Jinping, she added.
The dollar fetched 105.63 yen in afternoon trade, against 105.45 yen in New York late Tuesday.
Just minutes before the opening bell in Tokyo, China's official Xinhua news agency said Kim made his first-ever foreign trip as leader to meet China's president in Beijing.
Kim told Xi his nuclear-armed regime was willing to hold a summit with the US, and is "committed to denuclearisation", Xinhua said.
In Japan meanwhile, "political risks have been subdued" after an official at the heart of a cronyism and cover-up scandal said Tuesday Prime Minister Shinzo Abe was not involved in falsifying documents, said Shinichi Yamamoto, broker at Okasan Securities in Tokyo.
"But trading is expected to be range-bound for now as players are searching for fresh trading factors," Yamamoto told AFP.
In Tokyo, tech firms and automakers led the falls.
Panasonic plunged 5.09 percent to 1,565.5 yen and Tokyo Electron, a major maker of chip-making equipment, lost 4.43 percent to 19,590 yen, while telecom carrier and IT investor SoftBank dropped 4.01 percent to 7,922 yen.
Nissan fell 2.69 percent to 1,099.5 yen and Toyota slid 0.42 percent to 6,862 yen, with Subaru down 2.44 percent at 3,475 yen.
In New York, the Dow closed 1.4 percent lower while the tech-rich Nasdaq dropped 2.9 percent.
Tesla shares plunged 8.22 percent as the National Transportation Safety Board announced it is investigating the March 23 fatal accident.