The Bank of Japan's Tankan report -- a quarterly survey of about 10,000 companies -- showed a reading of 24 among major manufacturers in its March survey against 26 in the December report.
Confidence fell among chemicals, electrical machinery makers and other exporters as the average predicted yen rate for the business year from April appreciated to 109.66 to the dollar in the latest survey from above 110 yen in December.
The sentiment among iron and steelmakers tumbled to 10 from 19 as the US moved to impose hefty tariffs on steel and aluminium imports.
A strong yen is negative for Japanese exporters as it makes their products less competitive abroad and erodes profits when repatriated.
The latest Tankan, however, hardly made economists pessimistic about the world's third largest economy, with the headline index still close to the highest level in over a decade.
Economists argue Japan is on a solid recovery path on the back of a global economic recovery, with investments linked to the Tokyo 2020 Olympics also giving the economy a shot in the arm.
"The yen's appreciation dented corporate confidence temporarily," said UBS economist Takuji Aida.
"But the underlying sentiment is solid, as seen in their plans to accelerate spending on plants and equipment," he said.
"Weak stock prices (since February) have not prevented them from stepping up capital spending," he said.
The index for non-manufacturers also fell to 23 from 25 but Aida said it would also improve going forward, with rises in wages starting to translate into higher consumer spending.
"Readings in the next Tankan will likely turn up," he told AFP.
The Tankan report, the broadest indicator of how Japan Inc is faring, marks the difference between the percentage of firms that are upbeat and those that see conditions as unfavourable.
Due to a triennial reshuffling of companies surveyed, the confidence for the December 2017 quarter was recalculated with the new base in order to make a comparison.
The latest survey came after a batch of solid economic data released on Friday, which showed factory production picked up in February while the jobless rate stayed low.
Japan's economy expanded by 0.4 percent in the last quarter of 2017 as it notched up its longest period of expansion since the "bubble" days of the 1980s.
The stock market kept soaring in January but took a plunge in February on fears that the US central bank will embark on a sharper pace of interest-rate rises than previously expected to head off inflation.
"Looking forward, we do not see a deterioration" Katsunori Kitakura, lead strategist at Sumitomo Mitsui Trust, said in a commentary.
"Business sentiment may see some downward pressure until Q2 2018, but should recover towards year-end" as the global economy grows, he said in a commentary ahead of the release of the latest Tankan.
Prime Minister Shinzo Abe has been trying to boost the economy with a mix of aggressive monetary easing and huge government spending along with reforms to the economy.
But it has failed in its goal of shrugging off the deflation that has plagued Japan for years and held back growth.