It leaves English cricket searching for both a new coach and captain ahead of next month's series against Pakistan.
The England and Wales Cricket Board (ECB) confirmed on Sunday that McCullum will remain in charge of England's white-ball teams, but decided a change was needed in the Test setup following a poor run of results and an internal review after a disappointing Ashes campaign and home series defeat to New Zealand.
England have lost seven of their last nine Tests, including a 4-1 Ashes defeat in Australia and a 2-1 home series loss to New Zealand - their first home defeat in a Test series of three or more matches in 14 years. The recent struggles have also been accompanied by a series of off-field controversies involving players and team discipline.
"I've absolutely loved coaching the Test side and I'm incredibly proud of what we've achieved together," McCullum said.
"There've been some unbelievable highs and a few tough days along the way, but that's all part of taking on a challenge like this.
"Of course I'm gutted not to be continuing, but I respect the decision. My focus now is on giving everything I've got to the white-ball teams and helping England keep moving forward."
ECB chief executive Richard Gould said the decision was made with an eye on the future.
"Brendon breathed new life into England's Test side during an exciting period that delivered some memorable victories, and we're grateful for everything he has given to the role. We now believe that the time is right to make a change for the Test team as we target victory in the Ashes next (English) summer," said Gould.
Managing director Rob Key, who appointed McCullum in 2022, will remain in his role despite overseeing the review.
McCullum's appointment alongside Stokes transformed an England side that had won only one of its previous 17 Tests. Their aggressive "Bazball" philosophy produced immediate success, with England winning 10 of their first 11 Tests, including memorable victories over New Zealand, India and Pakistan, where they completed a historic 3-0 clean sweep.
England also recovered from 2-0 down to draw the 2023 home Ashes 2-2 and famously won the opening Test in India in 2024 before the series slipped away in a 4-1 defeat.
However, the momentum faded. England won only 17 of their final 38 Tests under McCullum while losing 19, failing to win any of their marquee five-Test series against Australia or India.
The disastrous Ashes tour in Australia proved a turning point. England arrived with an inexperienced squad and limited preparation before suffering a heavy 4-1 defeat despite claiming their first Test victory in Australia for 14 years in Melbourne. The tour was also overshadowed by criticism of the team's preparation and allegations of a drinking culture.
Those concerns persisted after Harry Brook was involved in an incident with a nightclub bouncer in New Zealand, while Stokes and Gus Atkinson later missed a Test against New Zealand after breaching a team curfew following a night out in London.
The fallout continued when Stokes announced his retirement after the New Zealand series, ending the leadership partnership that had defined England's Test revival over the past four years.
Attention now turns to finding his successor. Harry Brook is widely regarded as the leading candidate to replace Stokes as Test captain, although the appointment could depend on who is chosen as England's next head coach.
Among the possible coaching candidates are former England coach Andy Flower, Glamorgan's Richard Dawson, Surrey's Gareth Batty, Durham coach Ryan Campbell and former Australia coach Darren Lehmann, while Andrew Flintoff has previously ruled himself out despite leading the England Lions.
Although his Test tenure has ended, McCullum remains firmly in charge of England's white-ball programme. After a disappointing Champions Trophy campaign earlier this year, he recently guided England to a 4-0 T20 series sweep over world champions India, lifting them back to the top of the international rankings, with his contract running until the 2027 50-over World Cup.
Jos Buttler hit a swashbuckling century as England continued to dominate India, the hosts beating the visitors by 56 runs in the final game in Southampton to win the five-match T20I series 4-0.
The proposed Euro Nations Cup has encountered a significant obstacle after the England and Wales Cricket Board (ECB) declined to support plans for the tournament's showpiece match to be staged at Lord's.
The International Cricket Council has raised concerns with the England and Wales Cricket Board over the handling of Ben Stokes' international retirement announcement.
England fast bowlers Josh Tongue and Jofra Archer shared seven wickets to condemn India to their heaviest ever T20I defeat as the tourists went down by 125 runs in the third game of series at Trent Bridge.
