Anderson, who started this match with 399 Test wickets, needed just eight balls to reach the landmark on Friday when he had opener Guptill well-caught for a duck by Ian Bell, who held a sharp chance above his head at second slip off a classic swinging delivery.
Lancashire paceman Anderson, 32, achieved the milestone in his 104th Test.
No sooner had Anderson struck then rain, which washed out the morning session, returned to force a 10-minute stoppage.
But when the players returned, Anderson soon had his 401st Test wicket when Kane Williamson, who scored a century in England's 124-run first Test win at Lord's, was caught behind by wicket-keeper Jos Buttler for a duck.
Anderson had taken two wickets for no runs in three balls to leave New Zealand two for two in the third over of a match the tourists had to win to square the two-match series at 1-1.
England's spearhead bowler is now one of eight pacemen in a '400-club' list topped by retired Sri Lanka off-spin great Muttiah Muralitharan, who took a Test record 800 wickets in 133 matches from 1992-2010.
Most Test wickets (wickets, bowler, country, Tests, average, career span):
800 Muttiah Muralitharan SRI/ICC 133 22.72 1992-2010
708 Shane Warne AUS 145 25.41 1992-2007
619 Anil Kumble IND 132 29.65 1990-2008
563 Glenn McGrath AUS 124 21.64 1993-2007
519 Courtney Walsh WIS 132 24.44 1984-2001
434 Kapil Dev IND 131 29.64 1978-1994
431 Richard Hadlee NZL 86 22.29 1973-1990
421 Shaun Pollock RSA 108 23.11 1995-2008
414 Wasim Akram PAK 104 23.62 1985-2002
413 Harbhajan Singh IND 101 32.37 1998-
405 Curtly Ambrose WIS 98 20.99 1998-2000
401 James Anderson ENG 104 29.37 2003-
Note: Anderson's average prior to start of his ongoing 104th Test, against New Zealand at Headingley.