The Nikkei 225 index at the Tokyo Stock Exchange, which closed at its highest point since August 5 on Wednesday, fell 24.45 points to 9,235.89, while the Topix index of all first-section issues fell 0.33 percent or 2.67 points to 800.29.
"The market may take a break after reaching the psychologically important 9,250 mark (on Wednesday)," said Daiwa Securities senior market analyst Yumi Nishimura.
The market looks overbought in the short-term, with autos and financials ripe for profit-taking after rising sharply in the previous session, she told Dow Jones Newswires.
US stocks ended in the red Wednesday amid renewed concerns over Greece's debt crisis. The Dow Jones Industrial Average fell 0.76 percent to finish at 12,780.95.
Mizuho Securities senior technical analyst Yutaka Miura noted that the yen had strengthened against the euro and was also negatively impacting the Nikkei.
The euro fell to 102.12 yen in Tokyo midday trading from 102.47 yen in New York late Wednesday.
Eurozone finance chiefs told Greece Wednesday it must accept tough surveillance if it is to close a stalled bailout deal by next week, despite meeting key hurdles.
Eurozone finance ministers will be set to take "all the necessary decisions" on Greece on Monday after Athens met conditions for a bailout, said Luxembourg Prime Minister Jean-Claude Juncker, who heads the group of ministers.
Olympus shares fell 1.30 percent to 1,287 yen as investors braced for the arrests of former president Tsuyoshi Kikukawa and two other executives over their roles in the company's huge cover-up scandal.
Japanese media reported the three were central in a scheme to shift $1.7 billion of losses off the camera-maker's balance sheet and had been taken into custody for questioning.