The benchmark Nikkei index, which on Tuesday fell nearly five percent in the worst loss since the November 2016 election of Donald Trump as US president, rose 0.16 percent or 35.13 points to close at 21,645.37.
The broader Topix index gained 0.37 percent or 6.50 points to 1,749.91.
The Nikkei index opened sharply higher, surging nearly 3.5 percent at one point as investors bought on dips following the recent slumps.
"If the New York Dow stops falling, foreign selling of Japanese stocks will likely cease," said Masayuki Kubota, chief strategist at Rakuten Securities.
"Given that the Japanese economy and corporate earnings are good, it is highly likely that 21,078 will be the bottom of the latest crash," he said in a note, referring to the Nikkei's intraday low on Tuesday when the key index was down more than seven percent.
But early gains were gradually eroded.