Graceful Shutdown and Connection Draining in Node and Express — SIGTERM, server.close(), Keep-Alive Drain, and the Readiness-Flip Pattern

server.close() does not do what its name says, and the gap is where a dropped request falls through.

§ IFrame

Node makes graceful shutdown look easy and then hides three traps in the details. The easy part: Node delivers SIGTERM to your process as an ordinary event, and you can listen for it the same way you listen for an HTTP request. The hard part: the obvious shutdown code — catch the signal, call server.close(), exit — drops requests in production, and it drops them in ways that pass every local test because the traps only spring under real traffic.

Today's Ops lesson named the five-beat drain: catch the signal, flip readiness to false, wait out the router propagation delay, stop the listener and drain in-flight work under a timeout, then exit. That contract is language-agnostic. This lesson writes it in Node and Express, and the writing is where the runtime teaches you what the contract left implicit. Node's single-threaded event loop, Express's layering on the core http server, and HTTP keep-alive each change what the five beats actually do.

The keep-alive trap alone is worth the lesson. server.close() does not do what its name suggests, and the gap between what you think it does and what it does is exactly the gap a dropped request falls through.

§ IILanguage Idiom — Signals, the Event Loop, and What server.close() Actually Does

Node's shutdown model rests on three runtime facts.

Signals are events on the process object. process.on('SIGTERM', handler) registers a listener exactly like any other event. When the orchestrator sends SIGTERM, Node schedules the handler on the event loop. This is the entry point for the entire drain. The default behavior, absent a handler, is to terminate the process immediately — which is the naive deploy. Installing a handler is the act of taking control of your own death.

The event loop will not exit while work is pending, and that is the lever. Node exits when the event loop has nothing left to do — no open listeners, no pending timers, no active handles. This is why a clean drain does not need a forced exit: once you close the listener and the last in-flight request finishes, the loop empties on its own and the process exits naturally. The drain timeout exists only for the pathological case where some handle never closes.

server.close() stops accepting, but waits for every existing connection — including idle ones. Here is the trap. The core http.Server.close() method stops the server from accepting new connections and invokes its callback only when all existing connections have ended. With HTTP keep-alive — which Express enables by default, and which every modern client uses — a connection does not end when a response is sent. It stays open, idle, ready for the next request. So server.close() waits for connections that will never close on their own, the callback never fires, the drain hangs until the timeout, and the orchestrator eventually kills the process hard. The naive drain becomes the naive deploy with extra steps.

§ IIICode Worked Example — The Five-Beat Drain in Express

Build the drain in the order the Ops contract demands. Start with the readiness flag and the health endpoint, because beat two depends on the load balancer being able to read the instance's readiness.

The readiness state is a single mutable flag the health route reports. While the flag is true the route returns 200; once shutdown flips it false the route returns 503, which is the signal the load balancer reads to stop routing new traffic.

const app = express();
let isShuttingDown = false;

app.get('/readyz', (req, res) => {
  if (isShuttingDown) {
    res.status(503).json({ status: 'draining' });
  } else {
    res.status(200).json({ status: 'ready' });
  }
});

const server = app.listen(8080);

The next block solves the keep-alive trap before it bites. Track every open socket as it connects, and mark which sockets are idle versus actively serving a request. When the drain reaches the listener-close beat, idle keep-alive sockets must be destroyed deliberately, because server.close() will wait on them forever otherwise.

The tracking uses the server's connection event to record each socket, and the request and response lifecycle to mark a socket busy while it serves and idle when it finishes.

const sockets = new Set();

server.on('connection', (socket) => {
  sockets.add(socket);
  socket.once('close', () => sockets.delete(socket));
});

app.use((req, res, next) => {
  req.socket._isHandling = true;
  res.once('finish', () => { req.socket._isHandling = false; });
  next();
});

Now the signal handler runs the five beats. Beat one is the handler firing without exiting. Beat two flips the readiness flag. Beat three waits out the router propagation delay while still serving. Beat four closes the listener, destroys idle keep-alive sockets so the close can complete, and waits for busy sockets to finish under a hard timeout. Beat five lets the empty event loop exit, with a forced exit only as the timeout backstop.

process.on('SIGTERM', async () => {
  isShuttingDown = true;

  await sleep(15000);

  server.close(() => process.exit(0));

  for (const socket of sockets) {
    if (!socket._isHandling) socket.destroy();
  }

  setTimeout(() => process.exit(1), 30000).unref();
});

Read the sequence against the contract. The fifteen-second sleep is beat three: the instance has already failed readiness, but it keeps serving because the routers have not all noticed yet. The server.close() call is beat four, and its callback is the natural clean exit when the last busy socket finishes. The loop over sockets is the keep-alive fix — idle sockets get destroyed so they do not hold the close open, while busy ones are left alone to finish their response. The unref'd timeout is the backstop: thirty seconds after the listener closes, if some request is still stuck, force the exit so the orchestrator is not left waiting. The .unref() matters because it lets the timer itself not keep the loop alive, so a clean drain exits the instant the real work is done rather than waiting for the backstop.

One more discipline the runtime demands: register the handler once, and guard against a second SIGTERM arriving mid-drain. A second signal should be ignored, not restart the sequence. The isShuttingDown flag already serves as that guard if the handler checks it first.

§ IVConnection to Today's Ops Lesson

This is the Ops lesson's five-beat drain, beat for beat, in Node. Beat one is process.on('SIGTERM', ...) without an exit call. Beat two is isShuttingDown = true driving the /readyz route to 503. Beat three is the deliberate sleep past the propagation delay. Beat four is server.close() plus the keep-alive socket destruction. Beat five is the event loop emptying, with the unref'd timeout as the hard ceiling the Ops lesson called the drain timeout.

What the runtime adds to the abstract contract is the keep-alive trap. The Ops lesson said "stop the listener and let in-flight work finish." In Node that single sentence hides the fact that server.close() cannot tell an idle keep-alive connection from a dead one, so the implementer must. The contract is correct at every scale; the runtime is where it gets teeth.

§ VPrior-Lesson Reach

The 06-17 Web lesson confined the Node runtime — JavaScript's permission model and the least-privilege module boundary. Graceful shutdown is the same posture pointed at the process lifecycle instead of the module graph: take explicit control of what the runtime does by default, because the default — exit on signal, wait forever on keep-alive — is wrong for production. The 06-14 React and Three.js lesson worked the render loop's frame budget, where the discipline was finishing the current frame cleanly before starting the next. The drain timeout is the same instinct on the server: finish the current request cleanly, but cap how long "current" is allowed to take, because one stuck handler cannot be allowed to hold the whole deploy hostage.

§ VIClosing

Node hands you SIGTERM as an event and server.close() as a method, and the two of them look like a complete graceful shutdown. They are not, because server.close() waits on idle keep-alive connections that never close themselves. The working drain tracks its sockets, flips a readiness flag the load balancer can read, waits out the routers on purpose, closes the listener while destroying idle sockets and sparing busy ones, and backstops the whole thing with an unref'd timeout. Five beats, one runtime, one trap that local testing will never reveal because local testing has no load balancer and no keep-alive pool.

Open your service's shutdown code. Find the server.close() call. If there is no socket-tracking set destroyed alongside it, your drain hangs on every deploy until the timeout kills it hard — and a hard kill drops exactly the requests the drain was supposed to save. Add the socket set first.

Paired lessons → Ops 01-Earth-DevOps/.../2026-06-21-zero-downtime-deploys-... · Cert Cert-Prep/Anthropic/2026-06-21-claude-code-as-the-operators-harness-...