The price of 10-year JGB futures fell 0.31 point, the biggest fall since April 27, to a three-week low of 151.52.
The yield on the cash 10-year JGB futures rose 3.0 basis points to minus 0.075 point, a three-week high.
The five-year yield rose 3.0 basis points to minus 0.195 percent, a seven-week high, while the two-year yield rose to a high of minus 0.225 percent at one point, the highest since April 6.
An auction of 1.1 trillion yen 20-year JGBs attracted decent bids, with the auction's tail, or the gap between average and lowest prices, shrinking to 0.15 from 0.21 in the previous auction.
The results helped to underpin the longer end of the market, with the 30-year yield falling 0.5 basis point to 0.350 percent. .
But that was not enough to turn around the overall market sentiment hit by sudden surge in expectations that the Fed could raise rates as soon as June.
The June Fed Funds rate futures are pricing in the chance of about a third of the move in June, compared to almost zero less than a week ago.