The tourists, seeking a series-levelling win after losing the first Test in Galle, took their second innings from the overnight 70-1 to 179-3 by lunch on the fourth day.
India have an overall lead of 266 with seven wickets in hand on a slow, wearing pitch that should favour their spinners on the last day.
Vijay made 82 and Rahane was unbeaten on 82, the pair sharing a second-wicket partnership of 140 that boosted the total after Lokesh Rahul had fallen off the fifth ball of the innings on Saturday evening.
Vijay hit four boundaries and two sixes, but missed his seventh Test century when he attempted to sweep off-spinner Tharindu Kaushal and was adjudged leg-before soon after the morning drinks break.
Kaushal then trapped captain Virat Kohli leg-before for 10, but Rohit Sharma saw out the session unbeaten on two.
India made a brisk start by adding 68 runs in the first hour, with Vijay slamming sixes off fast bowler Dushmantha Chameera and Kaushal.
At the other end, Rahane reached his half-century by reverse-sweeping Kaushal for his fifth boundary.
India scored 393 in their first innings after electing to bat and bowled out Sri Lanka for 306 to gain a lead of 87 runs.