WebSocket graceful shutdown in Node.js #

Every deploy produces a spike of 1006 abnormal closures, a burst of reconnects that lands on the remaining nodes all at once, and a handful of messages that clients swear they sent but the server never processed. The process received SIGTERM, Node’s default behaviour ran — which is to exit immediately — and every socket was torn down mid-frame without a close handshake. A WebSocket server needs an explicit shutdown sequence, because the defaults that work for short HTTP requests are wrong for connections that live for hours. This page builds one for a plain ws server, independent of the orchestrator; the Kubernetes side of the same problem is covered in draining WebSocket connections during deploys.

Root cause #

Three defaults combine to make shutdown abrupt. First, Node installs no handler for SIGTERM, so the signal terminates the process at once; the kernel then closes every file descriptor, which sends a FIN or RST on each TCP connection without any WebSocket close frame. Browsers report that as code 1006, which most reconnect logic treats as a network failure worth retrying immediately. Second, server.close() on the HTTP server stops new connections but does not close upgraded sockets — ws has detached them from the HTTP machinery — so a naive “close the server and wait” handler waits forever and is then killed anyway. Third, anything sitting in a socket’s bufferedAmount or in your own outbound queues is discarded when the process dies.

The result is a correctness problem as well as a load problem. Clients reconnect simultaneously to the surviving nodes, and messages in flight in either direction are lost unless your delivery guarantees layer replays them.

Server lifecycle during shutdown A server moves from serving to draining on SIGTERM, then to flushing once every client has been sent a going-away close, and exits once sockets close or a hard deadline passes. Server lifecycle during shutdown Serving accepting upgrades Draining reject new, close old Flushing wait for buffers Exited process ends SIGTERM all 1001 sent sockets closed hard deadline The hard-deadline edge must fire before the orchestrator's own kill timer, or you lose the chance to log what was cut off
Four states, and the deadline edge that guarantees the process exits on its own terms.

Resolution #

The handler below implements the four states. It stops the listener, rejects any upgrade that races in, closes existing clients with 1001 Going Away in small batches spread across a drain window, waits for close handshakes, and forces termination for anything still open at the deadline. The batching is what stops the deploy from turning into a reconnect storm on the other nodes.

import http from 'node:http';
import { WebSocketServer, WebSocket } from 'ws';

const DRAIN_WINDOW_MS = 20_000;     // spread closes over this period
const BATCH_COUNT = 10;             // number of close waves inside the window
const HARD_DEADLINE_MS = 25_000;    // must be < orchestrator grace period (e.g. 30 s)
const CLOSE_GOING_AWAY = 1001;

const server = http.createServer();
const wss = new WebSocketServer({ noServer: true });
let draining = false;

server.on('upgrade', (req, socket, head) => {
  if (draining) {
    // Refuse at the HTTP layer; the client's reconnect logic retries elsewhere.
    socket.write('HTTP/1.1 503 Service Unavailable\r\nRetry-After: 1\r\nConnection: close\r\n\r\n');
    socket.destroy();
    return;
  }
  wss.handleUpgrade(req, socket, head, (ws) => wss.emit('connection', ws, req));
});

function waitForClose(ws: WebSocket): Promise<void> {
  return new Promise((resolve) => {
    if (ws.readyState === WebSocket.CLOSED) return resolve();
    ws.once('close', () => resolve());
  });
}

async function shutdown(signal: string) {
  if (draining) return;              // second SIGTERM: already in progress
  draining = true;
  console.info({ signal, clients: wss.clients.size }, 'shutdown: draining');

  // Hard stop: whatever is left at the deadline is terminated, then we exit.
  const deadline = setTimeout(() => {
    for (const ws of wss.clients) ws.terminate();
    process.exit(0);
  }, HARD_DEADLINE_MS);
  deadline.unref();

  server.close();                    // stop accepting new TCP connections
  const clients = [...wss.clients];
  const perBatch = Math.ceil(clients.length / BATCH_COUNT);
  const gap = DRAIN_WINDOW_MS / BATCH_COUNT;

  const closes: Promise<void>[] = [];
  for (let i = 0; i < clients.length; i += perBatch) {
    for (const ws of clients.slice(i, i + perBatch)) {
      // The reason string lets clients tell a deploy from a failure.
      ws.close(CLOSE_GOING_AWAY, 'server restarting');
      closes.push(waitForClose(ws));
    }
    await new Promise((r) => setTimeout(r, gap));
  }

  await Promise.all(closes);         // close handshakes complete, buffers flushed
  clearTimeout(deadline);
  process.exit(0);
}

process.on('SIGTERM', () => void shutdown('SIGTERM'));
process.on('SIGINT', () => void shutdown('SIGINT'));
server.listen(8080);

ws.close() queues the close frame behind any data already buffered for that client, so a graceful close is also a flush: the client receives everything you sent before the close, in order. That ordering is what lets you deliver a final “server restarting, resume from sequence N” message before the close frame and have it arrive.

The client half matters just as much. A 1001 close should be treated as “reconnect soon, with jitter”, never as “reconnect now”, so that the batches you spread out on the server stay spread out on the network. The exponential backoff with jitter guide shows the client-side schedule.

Reconnect arrivals: abrupt exit vs staggered 1001 Reconnect attempts per second for 5000 dropped clients: a fixed retry delay lands every client in one second, full jitter spreads them across 8 seconds. Reconnect arrivals: abrupt exit vs staggered 1001 5.0k clients, 8s window — fixed delay vs full jitter Fixed delay Full jitter 0 2.0k 4.0k 6.0k 5.0k 1s 2s 3s 4s 650 5s 642 6s 645 7s 638 8s
Closing every socket at once concentrates reconnects into the first second; spreading closes over the drain window flattens the peak the surviving nodes must absorb.

Verification #

Test the handler locally before trusting it in a rollout. Start the server, connect a few hundred clients with a small script, then send SIGTERM and observe the close codes the clients receive:

node server.js & PID=$!
# 300 idle clients that print the close code and reason they receive
for i in $(seq 1 300); do
  node -e "const W=require('ws');const w=new W('ws://localhost:8080');w.on('close',(c,r)=>console.log(c,String(r)))" &
done
sleep 2; kill -TERM $PID
wait 2>/dev/null | sort | uniq -c
# Expect: 300 lines of "1001 server restarting", zero 1006

In production, graph close codes by deploy. A healthy rollout shows a block of 1001 closes spread across the drain window and almost no 1006. If 1006 still dominates, the process is being killed before the handler finishes — compare HARD_DEADLINE_MS with the orchestrator’s grace period, and check whether a process manager in the container (a shell entrypoint, for example) is swallowing the signal instead of forwarding it to Node.

A 30 s grace period, used well SIGTERM at zero, the listener closes at one second, close waves run from two to twenty seconds, and the process exits at twenty-three seconds, before a SIGKILL at thirty. A 30 s grace period, used well staggered closes SIGTERM received (0s) listener closed (1s) first 1001 wave (2s) last 1001 wave (20s) all closed, exit 0 (23s) SIGKILL would land (30s) Leave headroom between your own deadline and the platform's kill so the exit is always yours
Finish before the platform finishes you.

Operational checklist #

FAQ #

Why doesn’t server.close() close my WebSocket connections? #

server.close() only stops the HTTP server from accepting new connections and waits for existing HTTP requests to finish. Once a request is upgraded, ws owns the socket and the HTTP server no longer tracks it as active, so it will never be closed for you. You must iterate wss.clients and close them yourself.

Should I use 1001 or 1012 for a restart? #

1001 Going Away is the code the browser API documents and every client library understands, and it is the conventional choice for a server shutting down. 1012 Service Restart is registered with IANA and is more specific, but some clients treat unknown codes as errors. Use 1001 unless you control every client.

How long should the drain window be? #

Long enough to spread reconnects so no surviving node receives more than it can handshake per second, and short enough to fit inside the orchestrator’s grace period with a few seconds to spare. Twenty seconds inside a 30-second grace period is a common starting point; divide your connection count by the drain window to check the resulting reconnect rate.

What happens to messages the client sends during draining? #

Messages that arrive before the client processes your close frame are still delivered to your message handler, so keep handling them until the socket closes. Anything the client sends after receiving the close frame is dropped by the protocol, which is why clients should buffer unacknowledged messages and replay them after reconnecting.

Back to Connection Lifecycle & Heartbeats.