Fixing WebSocket CLOSE_WAIT Accumulation #

ss -tan | grep CLOSE_WAIT | wc -l returns 8,412 on a server holding 3,000 live connections. Memory looks fine, CPU looks fine, and then at some point in the afternoon the process starts throwing EMFILE: too many open files and refuses new upgrades. Restarting clears it for a day or two. This is the classic long-lived-socket file-descriptor leak, and CLOSE_WAIT is the kernel telling you precisely whose fault it is: yours.

Root cause #

CLOSE_WAIT is a TCP state with an unambiguous meaning. The peer sent a FIN — it is done sending — and the kernel acknowledged it and moved the connection to CLOSE_WAIT. The socket now sits there waiting for your application to call close() on its file descriptor so the kernel can send its own FIN and finish the teardown. A socket cannot leave CLOSE_WAIT on its own. There is no timeout, no reaper, no expiry: it stays until the process closes the descriptor or the process exits.

So a growing CLOSE_WAIT count always means the same thing — the application is not closing sockets whose peers have already gone away. In a WebSocket server, three patterns produce it:

The close handler throws. If your socket.on('close', …) handler raises before it reaches the cleanup — a null dereference on a session that was already deleted, an await on a Redis client that has been disconnected — the rest of the handler never runs. Anything you were about to release stays held.

A reference outlives the socket. The socket is in a Map keyed by user id, and the entry is only removed on a graceful logout message that a dropped client never sends. The ws object stays reachable, the underlying descriptor stays open, and the kernel keeps the entry in CLOSE_WAIT indefinitely.

Half-open connections nobody probes. The peer’s FIN arrives during a network event, but your server has no heartbeat, so nothing ever touches that socket again. It is idle, it looks healthy in your connection gauge, and it holds a descriptor forever.

The reason this is so often missed is that the connection count your application reports stays plausible. wss.clients.size reflects sockets ws still considers open, which is exactly the set that is leaking — so the gauge agrees with the leak instead of exposing it.

Resolution #

Two changes: make the close path unconditional and idempotent, and add a heartbeat that terminates sockets whose peers have stopped answering. The heartbeat matters because it is the only thing that discovers half-open connections at all.

import { WebSocketServer, WebSocket } from 'ws';

const HEARTBEAT_INTERVAL_MS = 30_000;
const sessions = new Map<string, WebSocket>();
const alive = new WeakMap<WebSocket, boolean>();

export function attach(wss: WebSocketServer): void {
  wss.on('connection', (socket, req) => {
    const sessionId = String(req.headers['sec-websocket-key']);
    sessions.set(sessionId, socket);
    alive.set(socket, true);

    socket.on('pong', () => alive.set(socket, true));

    // ONE cleanup path, registered on 'close', which ws emits for every ending:
    // clean close, abnormal close, error, and terminate(). 'error' alone is not
    // enough — an errored socket still emits 'close' afterwards.
    socket.on('close', () => {
      try {
        sessions.delete(sessionId);          // drop the strong reference FIRST
      } finally {
        // Anything that can throw goes after the reference is already gone, so a
        // failure in downstream cleanup can never strand the descriptor.
        void releaseDownstream(sessionId).catch(() => {});
      }
    });

    // Never let a listener throw into the socket's own event loop turn.
    socket.on('error', (err) => {
      console.error({ sessionId, err: err.message }, 'socket error');
      socket.terminate();                    // destroys the fd immediately
    });
  });

  const timer = setInterval(() => {
    for (const socket of wss.clients) {
      if (alive.get(socket) === false) {
        // The peer did not answer the last ping. close() would wait for a
        // handshake that is not coming; terminate() releases the fd now.
        socket.terminate();
        continue;
      }
      alive.set(socket, false);
      socket.ping();
    }
  }, HEARTBEAT_INTERVAL_MS);

  wss.on('close', () => clearInterval(timer));
}

async function releaseDownstream(sessionId: string): Promise<void> {
  // presence keys, subscriptions, rate-limit buckets — everything keyed by session
}

The finally block is the load-bearing line. Deleting the map entry before any awaitable work means that even if releaseDownstream rejects, the socket is unreachable and the descriptor is closed. The ordering is the fix; the error handling is just hygiene.

terminate() versus close() is the other decision. close() is correct when the peer is still listening — it sends a close frame and waits for the reply. For a peer that has already sent FIN, or one that has stopped answering pings, there is nothing to negotiate with, and only terminate() (which calls destroy() on the underlying socket) releases the descriptor immediately.

Four layers hold a reference to the same descriptor, and the leak is always the topmost one that nobody released.

Verification #

Prove the leak, then prove the fix, using the kernel’s own accounting rather than your application’s.

# How many sockets are stuck waiting on YOUR close(), and who owns them
ss -tan state close-wait | wc -l
ss -tanp state close-wait | head -20

# File descriptors held by the process against its limit
ls /proc/$(pgrep -f 'node .*server.js')/fd | wc -l
cat /proc/$(pgrep -f 'node .*server.js')/limits | grep 'open files'

# Node's own view — if this disagrees with ss, the leak is in your bookkeeping
node -e "console.log(process.report.getReport().libuv.filter(h=>h.type==='tcp').length)"

A healthy server shows a CLOSE_WAIT count in the low tens that rises and falls, never a monotonic climb. After deploying the heartbeat, watch the count over one full heartbeat interval: it should fall to near zero as half-open sockets are terminated. If it does not, the sockets are still referenced somewhere — check for a second Map, an array of message history, or a closure capturing the socket in a pending promise.

Operational checklist #

• Register cleanup on close, never only on error — ws emits close for every ending, including errors and terminations.
• Delete the socket from every map or set before any await, so a downstream failure cannot strand the reference.
• Wrap the close handler so it cannot throw; an exception there is what turns one bad session into a permanent descriptor leak.
• Add a heartbeat sweep and call terminate() — not close() — on sockets that missed their pong.
• Clear the heartbeat interval on server shutdown so tests and rolling restarts do not leak timers.
• Export a ws_close_wait_sockets gauge scraped from ss, and alert on a rising trend rather than an absolute value.
• Raise the process nofile limit to a real number (65536+) and fix the leak — the limit buys time, not safety.
• Rollback: the heartbeat is additive and can be disabled by clearing one interval if it turns out to be terminating healthy sessions.

FAQ #

Does raising the file-descriptor limit fix this? #

No. It postpones the crash and makes it worse when it arrives, because more connections are lost at once. Raise nofile to a sane value regardless — 65536 is a reasonable default for a socket server, and the shipped default of 1024 is far too low — but treat a rising CLOSE_WAIT count as a bug in your close path, because that is exactly what the kernel is reporting.

What is the difference between CLOSE_WAIT and TIME_WAIT? #

They sit on opposite sides of the close. TIME_WAIT is on the side that closed first, is bounded (typically 60 seconds on Linux), and clears itself; large numbers of it are usually normal. CLOSE_WAIT is on the side that has not closed yet, is unbounded, and never clears itself. A pile of TIME_WAIT is a tuning question. A pile of CLOSE_WAIT is an application bug.

Why does my connection gauge look normal while sockets leak? #

Because the gauge is derived from the same structure that is leaking. wss.clients.size counts sockets ws believes are open, and a half-open socket qualifies. Compare against the kernel: a ws_close_wait_sockets gauge sourced from ss will diverge from your application count long before anything else breaks, which is why WebSocket observability and monitoring recommends exporting both.

Does a load balancer make this better or worse? #

Worse, subtly. Behind a proxy such as an ALB or nginx, your peer is the proxy, and proxies close upstream connections aggressively on their own idle timeouts. That produces a steady trickle of FINs your server must handle correctly — so a close-path bug that would take weeks to matter with direct clients surfaces in hours. Make sure the proxy timeout exceeds your heartbeat interval, as covered in configuring nginx for WebSocket upgrades.

Do I still need this if I use Socket.IO? #

Yes. Socket.IO runs its own heartbeat at the Engine.IO layer, which covers the half-open case reasonably well, but your own per-connection bookkeeping is still yours to clean up on disconnect. The Map-that-outlives-the-socket pattern is framework-independent, and it is the most common of the three causes.

Related #

• Connection Lifecycle & Heartbeats — the lifecycle states and the sweep this fix rides on.
• Implementing WebSocket Ping/Pong in Node.js — the heartbeat implementation in full.
• WebSocket Close Codes Explained — which code to send when you do close deliberately.
• Fixing Slow Consumer Memory Growth — the other resource a stuck socket holds.
• WebSocket Observability & Monitoring — exporting kernel-side socket state next to application counts.

Back to Connection Lifecycle & Heartbeats