Rahane and Vijay put on 112 for the first wicket after India were put in to bat, and although the tourists could not fully capitalise on that platform, they were still able to post 271 for eight.
Vijay top-scored with 72, his first half-century in ODI cricket, while Rahane made 63.
Kumar produced a devastating opening spell when he took the wickets of Hamilton Masakadza and the in-form Elton Chigumbura.
Chamu Chibhabha made 72, but the hosts faded after he was run out and they were eventually dismissed for 209, with Kumar finishing with figures of 4 for 33.
India won the opening game on Friday in a much tighter affair, prevailing by six runs after Chigumbura had taken the hosts agonisingly close to victory with a superb 104 not out.