The Nikkei 225 index at the Tokyo Stock Exchange, which on Wednesday ended at its highest point since August 5, fell 22.24 points to 9,238.10. The Topix index of all first-section issues fell 0.34 percent or 2.71 points to 800.25.
"Stocks are short-term overbought, and this is continuing to invite selling for now," Masatoshi Sato, senior strategist at Mizuho Investors Securities, told Dow Jones Newswires.
On Wall Street stocks ended in the red amid concerns over Greece's debt crisis. The Dow Jones Industrial Average fell 0.76 percent to finish at 12,780.95.
Eurozone finance chiefs told Greece Wednesday it must accept tough surveillance if it is to close a stalled bailout deal by next week, despite meeting key hurdles.
The development sent the euro lower, with the common unit standing at $1.3024 and 102.17 yen in Tokyo afternoon trade, compared with $1.3065 and 102.47 yen in New York late Wednesday.
However, the Tokyo bourse enjoyed little support from the relatively weaker yen, thanks to the Bank of Japan's announcement this week that it would further loosen monetary policy as it tries to battle stubborn deflation.
The dollar traded at 78.41 yen in Tokyo, compared with 78.43 in New York.
"There is not a great deal of fear of much more yen strengthening. Rather, signs of yen weakening, especially versus the dollar, remain an important potential catalyst for bulls," Sato said.
Major banks and shippers lost, but major exporter shares managed to post gains.
Mitsubishi UFJ Financial Group fell 1.71 percent to 401 yen and Mitsui OSK Line plunged 2.75 percent to 353 yen.
But Nintendo added 0.71 percent to 11,310 yen, Sony rose 0.88 percent to 1,589 yen and Canon rose 0.14 percent to 3,535 yen.
Olympus fell 2.37 percent to 1,273 yen as prosecutors arrested former president Tsuyoshi Kikukawa and two other executives over their roles in a company's $1.7 billion cover-up at the camera-maker.