After resuming Saturday's third day on 223-2, England collapsed to 354 all out, to ultimately concede an 84-run first-innings deficit.
Nathan Smith claimed 4-91 and Will O'Rourke took 3-53 as the tourists exploited increasing seam movement and uneven bounce.
Harry Brook top-scored with 58, while Jacob Bethell added 74, but England lost wickets regularly after Joe Root, Bethell and Jamie Smith all fell in the opening six overs.
Zak Foulkes, playing as a concussion substitute, impressed with 3-35, including the key wickets of Ben Stokes and Brook.
England responded strongly with the ball as Jofra Archer produced a fiery spell to remove Tom Latham and Devon Conway, reducing New Zealand to 12-2 and briefly reviving the hosts' hopes.
However, Rachin Ravindra's unbeaten 60 and Daryl Mitchell's 26 not out guided New Zealand to 120-3 at stumps with an unbroken 69-run partnership.
Gus Atkinson claimed the other wicket, dismissing Henry Nicholls, as England were left facing an uphill battle heading into day four.