While morning trade was supported by gains in US stocks, selling pressure in stock index futures took hold in the afternoon, with the Nikkei briefly dropping nearly 400 points, reports Xinhua news agency.
The benchmark Nikkei stock index, the 225-issue Nikkei Stock Average, declined 157.23 points, or 0.4 per cent, to close at 39,376.09.
The Nikkei has recently been struggling to rise past the 39,800 level, prompting cautionary sell-offs as investors eyed the market's resistance to further gains.
In contrast, the Tokyo Stock Price Index (TOPIX) rose for the first time in three days, closing 1.84 points, or 0.07 per cent higher at 2,741.52.
Of the listed stocks on the top-tier Prime Market, 784 declined, 800 rose, and 61 remained unchanged.
