There have been 6 other zions. And the One always chose the other door. The one leading the salvation of zion. Meaning the One and a few survivors repopulated zion, while the Matrix reorganized itself with one big scandisk sorta.
Zion is just the minority of humans who reject the matrix and cant be controled. But this time, this Neo decided things were gonna be different, and chose love rather then salvation.
But in the end, everything still turned out right, cuz now theres an unsetteled peace between the surviving humans and the machines. There were rumors of maybe a 4th Matrix movie, cuz Neo will respawn again since hes the One and the human varible in the Matrix equation.