The halt is the machine choosing safety. The recovery is the operators keeping it.
A chain that produces a block every six seconds has one failure the validator most fears: the block that never comes. The screen shows the same height for a minute, then five, then an hour. Consensus has stopped. No new proposal, no new commit, no path forward through the software that was running when the machine stood still. This is the chain halt, and it is a different animal from the governed upgrade.
The governed upgrade is planned. Governance passes a proposal, the module writes an upgrade plan at a target height, the chain halts on purpose, every operator swaps to a staged binary, and the chain resumes. That path was the 2026-05-30 lesson. The chain-halt recovery path begins where no one planned to stop. A consensus bug, an apphash mismatch across implementations, a state that no available binary can advance. The chain is down, and because it is down, governance cannot act. The chain cannot vote itself back to life.
The recovery is off-chain and social before it is technical. Operators coordinate outside the halted machine, agree on a fix, export the last good state, transform it into a new genesis file, and relaunch a new chain that inherits the old one's balances and history. This lesson names that path as an operational discipline: the export-migrate-relaunch sequence.
A blockchain is a replicated state machine. Every validator runs the same deterministic program over the same ordered log of transactions and must arrive, byte for byte, at the same state. The consensus engine's job is to agree on the order; the application's job is to agree on the result. When those two agree across two-thirds of the stake, a block commits. When they cannot, the chain halts rather than fork. A halt is the consensus engine choosing safety over liveness, which is the correct choice.
The halt cannot be cleared by the running binary, because the running binary is the thing that cannot advance. Something must change: a patched consensus rule, a corrected state-migration, a fix to the determinism bug that split the validators. And the changed program cannot simply pick up from the halted database, because the halted database may itself carry the corruption or the disputed state that stopped the chain. The clean path is to take the last agreed state, lift it out of the stopped machine, and plant it as the first block of a new one.
That lifting is a state export. The chain writes its entire current state (every account, every balance, every delegation, every governance record) into a single genesis JSON, and a new chain reads that JSON as block zero. Kleppmann's account of schema evolution in Designing Data-Intensive Applications (Ch. 4, pp. 141–150) names the constraint that governs the transform: the exported state and the new binary must be compatible across the encoding boundary. Old data must read correctly under new code, and where the new code expects fields the old export lacks, the migration must supply them. Rolling upgrades keep old and new versions running side by side; a genesis restart is the harder case where every node cuts over at once, so the compatibility must be exact before the first new block.
Three phases, each with a distinct actor, a distinct artifact, and a distinct failure mode.
Export the last good state at the height every honest validator agrees on. Failure mode: silent divergence. Defense: the checksum — one file, one hash.
Transform the export to the new binary's schema. Supply defaults for new required fields. Hand-edit chain-id and genesis_time. Every edit is a published claim.
Every validator starts the new binary against the agreed genesis at the shared genesis_time. Rehearse the cutover on a copy; the old database is the rollback.
One operator, or a small coordinated set, runs the export against the halted node's database at the last committed height:
gaiad export \
--height <last-committed-height> \
--home <node-home> \
> exported_state.jsonThe failure mode is silent divergence. If two operators export from two databases that disagreed at the halt height, their genesis files differ, and the relaunch splits before it starts. The defense is the checksum: every operator's export at the agreed height must hash to the same value. The genesis handoff is not complete until the network has one file and one hash.
gaiad genesis migrate v0.47 exported_state.json \
--chain-id <new-chain-id> \
> migrated_genesis.jsonA parameter the new module requires but the old state never held must be given a default in the migration. A record the old chain stored in a format the new chain cannot parse must be rewritten. The migrated genesis also carries deliberate human edits: a new chain-id, a fresh genesis_time, sometimes a patched parameter the halt demanded. Every hand-edit is a claim the whole network must accept, so every hand-edit is published and hashed with the rest.
The genesis carries a genesis_time; nodes that start before it wait, nodes that start after it join late. Consensus resumes only when validators holding more than two-thirds of the stake are online and signing.
The chain halted at height 4,208,115 on an apphash mismatch. A patched binary, v0.47.2, corrects the non-determinism. The recovery runs end to end.
gaiad status --home ~/.gaia 2>&1 | jq '.SyncInfo.latest_block_height'The number must match across the network. A validator reporting a higher height committed a block others did not; that validator's state is suspect and must not seed the export.
gaiad export \
--height 4208115 \
--home ~/.gaia \
> exported_4208115.json
sha256sum exported_4208115.jsonThe hash is posted to the coordination channel. When enough operators post the same hash, the export is canonical.
gaiad genesis migrate v0.47 exported_4208115.json \
--chain-id testchain-5 \
--genesis-time 2026-07-04T18:00:00Z \
> genesis_testchain-5.json
gaiad genesis validate genesis_testchain-5.jsonThe validate step parses the migrated genesis under the new binary's rules and rejects a malformed file before relaunch, not during it.
sha256sum genesis_testchain-5.json
cp genesis_testchain-5.json ~/.gaia/config/genesis.json
gaiad start --home ~/.gaiaEvery operator confirms the genesis hash matches the canonical value before copying it into place. The genesis-time set to 18:00 UTC gives the network a shared relaunch moment.
The chain upgrade coordination lesson (2026-05-30) documented the planned x/upgrade halt-and-resume: governance schedules a halt, operators swap binaries, the chain continues from the same database. Today's sequence is that lesson's emergency twin. The planned upgrade resumes the same chain from disk; the genesis restart abandons the disk and rebuilds from an export.
The governance operations lesson (2026-06-27) named the case where the chain routes its future through the validator set on-chain. The genesis restart is the case where it cannot. When consensus is down, the vote moves off-chain to the coordination channel, and the migrated genesis is where the off-chain agreement becomes the new chain's law.
The RPC and full-node infrastructure lesson (2026-06-09) built the state-sync and snapshot-restore discipline. Export is snapshot-restore's cousin. A snapshot captures state to catch a new node up to a running chain; an export captures state to seed a chain that is not running yet.
Today's Bash lesson encodes the sequence as an executable pipeline. Three guards carry the discipline: an export-edit-verify pipeline that runs export, migration, and validation in one traceable pass; a genesis checksum gate that refuses to install any genesis whose hash does not match the canonical value; and a synchronized relaunch guard that holds the start command until the genesis_time arrives. The checksum gate is the script form of §III's silent-divergence defense — the pipeline will not let an operator relaunch on the wrong genesis, because the wrong genesis is exactly how a recovery splits.
A chain halt is not the disaster. The disaster is the recovery run cold, with no agreed height, no shared hash, and no rehearsal, split into two chains by an operator who copied the wrong genesis at the wrong minute. The halt is the machine choosing safety; the recovery is the operators keeping it.
Hold the sequence. Export at the height everyone agrees on. Migrate across the encoding boundary so old state reads under new code. Publish the hash and relaunch on the shared clock, because on-chain time is the only clock the whole network shares. Rehearse the cutover on a copy before the copy is the chain.
Examine the sequence well. The chain you recover is the one you practiced recovering.