WebSocket Backpressure & Flow Control #

Your market-data service has run for six days. Memory climbs 40 MB an hour, always on the same two pods, and a heap snapshot shows tens of thousands of retained Buffer objects hanging off socket send queues. Nothing is leaking in the classic sense — every handler unregisters correctly, every closed socket is removed from the map. The problem is simpler and nastier: you are producing 800 updates a second for a client whose phone can absorb 200, and socket.send() never told you. TCP applied backpressure to the kernel, the kernel applied it to Node’s stream, and Node did the only thing it could — it queued your data in userspace and kept accepting more.

That queue is bufferedAmount, and it is the single most under-instrumented number in a real-time backend. This guide covers reading it, acting on it, and designing message flows that degrade instead of exploding: high/low watermarks, per-client token buckets, coalescing keyed state, and the explicit decision every real-time system must make — when the consumer cannot keep up, what do you throw away?

Prerequisites #

Backpressure work sits on top of a healthy connection lifecycle. You need heartbeats running so a socket that has stopped draining because it is dead, rather than slow, is reaped by connection lifecycle and heartbeats instead of accumulating a queue forever. You need WebSocket observability and monitoring in place, because every policy below is worthless if nobody can see how often it fires. And if you broadcast across more than one node, the fan-out path from Redis pub/sub fan-out multiplies whatever per-socket cost you fail to bound here by the number of sockets on the node.

The code assumes the ws package on Node 20+, TypeScript, and a per-connection session object you can hang counters off.

The measured shape of that failure is the point of the next chart: the gap between offered rate and drain rate is a straight line into your heap, and a watermark policy is what turns it into a plateau.

Core implementation #

The whole policy fits in one send helper. Every write in your application goes through it, and it makes three decisions in order: is the socket still writable, is this client over its rate budget, and is the queue already past the high-water mark? Only if all three pass does the payload reach send().

import { WebSocket } from 'ws';

// Named thresholds — never scatter magic numbers through handlers.
const HIGH_WATER_MARK_BYTES = 2 * 1024 * 1024;   // stop sending above this
const LOW_WATER_MARK_BYTES  = 256 * 1024;        // resume once drained below this
const KILL_WATER_MARK_BYTES = 8 * 1024 * 1024;   // hopeless: close the socket
const TOKENS_PER_SECOND     = 60;                // sustained per-client message budget
const BUCKET_CAPACITY       = 120;               // burst allowance

type Priority = 'critical' | 'stateful' | 'ephemeral';

interface Session {
  socket: WebSocket;
  tokens: number;
  lastRefillMs: number;
  paused: boolean;
  /** Latest value per key — a coalescing buffer for `stateful` messages. */
  pending: Map<string, unknown>;
  dropped: number;
}

/** Refill the token bucket lazily; no timer per connection at 50k sockets. */
function takeToken(s: Session, now: number): boolean {
  const elapsedSec = (now - s.lastRefillMs) / 1000;
  s.lastRefillMs = now;
  s.tokens = Math.min(BUCKET_CAPACITY, s.tokens + elapsedSec * TOKENS_PER_SECOND);
  if (s.tokens < 1) return false;
  s.tokens -= 1;
  return true;
}

export function trySend(s: Session, key: string, payload: unknown, priority: Priority): boolean {
  if (s.socket.readyState !== WebSocket.OPEN) return false;

  const queued = s.socket.bufferedAmount;

  // 1. Hopelessly behind: this client will never catch up. Close it and let the
  //    client reconnect and resynchronise from a snapshot.
  if (queued > KILL_WATER_MARK_BYTES) {
    s.socket.close(1013, 'client too slow');     // 1013 = Try Again Later
    return false;
  }

  // 2. Above the high-water mark: only critical traffic gets through. Stateful
  //    updates collapse into `pending` (last value wins); ephemeral ones vanish.
  if (queued > HIGH_WATER_MARK_BYTES) {
    s.paused = true;
    if (priority === 'stateful') { s.pending.set(key, payload); return false; }
    if (priority === 'ephemeral') { s.dropped += 1; return false; }
  } else if (s.paused && queued < LOW_WATER_MARK_BYTES) {
    // 3. Drained back below the low-water mark: flush the coalesced state and resume.
    s.paused = false;
    for (const [k, v] of s.pending) s.socket.send(JSON.stringify({ type: 'state', key: k, value: v }));
    s.pending.clear();
  }

  // 4. Rate limit applies even when the queue is healthy: it protects the CPU
  //    cost of serialisation and framing, not just memory.
  if (priority !== 'critical' && !takeToken(s, Date.now())) {
    s.dropped += 1;
    return false;
  }

  s.socket.send(JSON.stringify({ type: 'msg', key, value: payload }));
  return true;
}

Two details matter more than the thresholds. First, the hysteresis between HIGH_WATER_MARK_BYTES and LOW_WATER_MARK_BYTES: a single threshold makes the socket flap between paused and resumed on every drained packet, and the flapping itself costs more than the traffic. Second, the pending map — for anything that represents current state rather than an event, the correct behaviour under pressure is to keep only the newest value per key. A price ticker that falls behind should never replay a queue of stale prices; it should jump to the current one.

Do not confuse this with the ws send() callback. The callback fires when the frame has been handed to the socket, which under load can be seconds after you called it, and queueing your own work behind it just moves the unbounded queue somewhere else.

Each socket therefore moves through four states, and the transitions — not the thresholds — are what you debug at 3 a.m. A socket that oscillates between flowing and paused many times a second has a hysteresis gap that is too narrow; a socket that reaches terminating regularly has a client that should be receiving snapshots rather than a stream.

Picking thresholds from measured traffic #

Every number in the table below should come from your own histograms, not from this page. The procedure takes an afternoon and it is the difference between a policy that protects you and one that fires constantly on healthy clients.

Start by recording bufferedAmount for every socket on each heartbeat sweep and exporting it as a histogram rather than a gauge — a gauge shows you the fleet average, which is always near zero and always useless. What you need is the p99 and the maximum across a full daily cycle, including whatever your worst hour looks like. In a typical dashboard workload the p99 sits in the low tens of kilobytes and the maximum spikes to a few hundred during reconnect waves; a high-water mark just above that maximum is the one that never fires on a healthy client.

Next, size the mark in time, not bytes. Divide your candidate threshold by the per-client output rate in bytes per second: 2 MB against a client receiving 40 KB/s is fifty seconds of backlog, which is far too generous — by the time you react, the user has been looking at stale data for the better part of a minute. Two to five seconds of backlog is the range where the queue is long enough to absorb a network hiccup and short enough that pausing still produces a fresh-looking UI when it resumes. For most dashboards that lands between 256 KB and 2 MB.

The kill mark answers a different question: at what point is a full reconnect and resynchronisation cheaper than catching up? Compute the cost of your snapshot — the bytes a fresh client downloads to render current state — and set the kill mark at roughly four times it. Once a client is that far behind, sending it the backlog costs more bandwidth than sending it the world, and the backlog leaves it stale for longer. This is why 1013 Try Again Later is the right close code: it tells a well-behaved client this is a capacity decision, not an error, and the reconnect logic should back off rather than retry immediately.

Finally, size the token bucket from client behaviour rather than server capability. Instrument inbound and outbound message counts per session, take the p99 over a week, and set TOKENS_PER_SECOND to roughly 1.5× that value with a burst capacity of double. Buckets sized from a server-side “what can we afford” number invariably throttle the power users you least want to throttle, while leaving abusive clients comfortably inside the limit.

Configuration reference #

Edge cases & gotchas #

bufferedAmount is bytes queued, not messages, and it lies about compression. With perMessageDeflate enabled, the value reflects pre-compression bytes for data still in the deflate pipeline, so a client on a slow link can look 3× worse than it is. Pick thresholds with compression in the state you actually ship.

Broadcast loops amplify by fleet size. wss.clients.forEach(c => c.send(payload)) with one slow client out of 20,000 is fine; with 5% slow clients on a node it is 1,000 unbounded queues growing at once. Always run the broadcast through trySend, and track the drop counter — a rising drop rate on one node is a much earlier signal than memory.

Closing a socket does not free its queue immediately. close() starts the closing handshake and the queued frames still drain (or wait for the close timeout). For a client that is genuinely stuck, socket.terminate() after a short grace period is the only thing that reclaims the memory now. Terminate is also the right call when bufferedAmount has not moved across two heartbeat intervals.

Node’s event loop is a backpressure signal too. If serialisation is your bottleneck rather than the network, bufferedAmount stays low while event-loop lag climbs. Watch perf_hooks monitorEventLoopDelay alongside the queue; the mitigations are different — sharding and cheaper serialisation, not watermarks. Binary framing, covered in message framing and serialization, often halves the CPU cost before any policy is needed.

Verification #

Confirm the policy is live before you trust it. Reproduce a slow consumer deterministically by connecting a client that never reads: open a raw TCP socket, complete the upgrade, then simply do not call recv. The kernel receive window closes, and your server-side queue starts growing within a second or two.

# Per-socket send queue depth on the server (Send-Q column, bytes)
ss -tn state established '( sport = :8080 )' | sort -k3 -nr | head -20

# Node heap growth attributable to buffers, sampled every 10s
node --expose-gc -e "setInterval(()=>console.log(process.memoryUsage().external),10000)"

Then assert on the metrics: ws_send_queue_bytes (a gauge you set from bufferedAmount on each heartbeat sweep) should show a bounded plateau under load, and ws_messages_dropped_total should be non-zero and stable, not accelerating. If drops accelerate while queue depth is flat, your token bucket is too small for real traffic; if queue depth grows while drops stay zero, the policy is not on the hot path at all.

Guides in this area #

• Handling WebSocket bufferedAmount Backpressure — reading the queue depth correctly, choosing watermarks from your own traffic, and wiring the pause/resume cycle.
• Rate Limiting WebSocket Messages Per Client — a token bucket for inbound and outbound traffic that survives 50,000 sockets without 50,000 timers.
• Fixing Slow Consumer Memory Growth — diagnosing the heap snapshot that says “retained by WebSocket send queue” and the terminate-versus-close decision.

FAQ #

Does bufferedAmount work the same in the browser and in the ws package? #

Close enough to reason about, but not identical. In browsers, WebSocket.bufferedAmount is defined by the HTML standard as the bytes queued by send() and not yet transmitted, and it decrements as the network drains. In ws on Node, it is the sum of the socket’s pending writes plus anything queued inside the sender, and it includes framing overhead. Both are safe to threshold on; neither is precise enough to use as an exact byte accounting.

Should I use a high-water mark or just close slow clients? #

Both, at different thresholds. A brief spike above the high-water mark is normal — a client switching networks, a tab waking up — and closing on it produces a reconnect storm that costs more than the queue. Reserve closing for the kill watermark, and always close with code 1013 so a well-behaved client backs off rather than reconnecting instantly. The auto-reconnection strategies guide covers the client half of that contract.

Does this work behind AWS ALB? #

Yes, and the ALB makes it more important, not less. The load balancer terminates the client TCP connection and opens its own to your target, so the receive window you are pushing against belongs to the ALB, not the browser. The ALB buffers a bounded amount and then stops reading from your target — which shows up as your bufferedAmount growing exactly as it would with a direct slow client. Thresholds and policy are unchanged; see configuring AWS ALB for WebSocket sticky sessions for the surrounding setup.

What changes for Socket.IO versus raw ws? #

Socket.IO layers its own packet encoder and an internal buffer on top of the engine’s transport, so socket.conn.transport is where the real queue lives; the Socket object you hold does not expose bufferedAmount directly. The policy shape is the same, but you read socket.conn.transport.writable and the engine’s writeBuffer.length instead of a byte count, which is coarser. If backpressure is a first-class concern, raw ws gives you the number you need.

Is dropping messages ever acceptable? #

For anything that represents current state, dropping is not just acceptable — it is correct, because delivering a stale value later is worse than never delivering it. For anything that represents an event with meaning (an order, a chat line, an ack), dropping is a correctness bug and you must instead apply backpressure upstream or persist and replay. That is exactly the boundary drawn in message delivery guarantees: classify every message type before you write the policy.

Related #

• Connection Lifecycle & Heartbeats — reaping sockets that stopped draining because they are dead rather than slow.
• WebSocket Observability & Monitoring — the queue-depth gauge and drop counter that make every threshold here reviewable.
• Message Delivery Guarantees — deciding which message classes may be dropped and which must be redelivered.
• Message Framing & Serialization — cutting bytes and CPU per message before you need a drop policy at all.
• Redis Pub/Sub Fan-Out for WebSockets — the fan-out path that multiplies any unbounded per-socket cost by the sockets on the node.

Back to Backend WebSocket Connection Management