Upul Tharanga was batting on 65 and Lahiru Thirimanne was yet to open his account.
Sri Lanka trail South Africa by 351 runs with seven wickets in hand.
The second and final Test will be played in Colombo from July 24-28.