The Nikkei 225 index lost 2.01 percent, or 436.87 points, to close at 21,341.74, while the broader Topix index was down 1.72 percent, or 27.42 points, at 1,568.87.
"The Japanese market is falling sharply, pressured by sell-offs in US shares and a higher yen," Toshiyuki Kanayama, senior market analyst at Monex, said in a commentary.
Wall Street stocks tumbled Wednesday after payrolls firm ADP estimated the US added 135,000 private-sector jobs in September. The figure was below market expectations, bolstering anxiety over the US economy.
"The market is now focusing on US unemployment figures on Friday and US-China talks next week," said Shinichi Yamamoto, a broker at Okasan Securities in Tokyo.
The dollar fetched 107.18 yen in Asian afternoon trade, against 107.16 yen in New York late Wednesday.
In Tokyo, shares were lower across the board, with Toyota dropping 2.47 percent to 6,982 yen and Honda dipping 2.67 percent to 2,742.5 yen.
Sony fell 0.83 percent to 6,148yen and Hitachi was off 1.64 percent at 4,006 yen.
Kansai Electric was down 1.06 percent at 1,252.5 yen after the company admitted two executives who were responsible for its nuclear business both received more than 100 million yen ($929,000) in gifts from a former official of a town hosting one of its nuclear plants.