Rate limiting WebSocket handshakes #

A buggy client release reconnects in a tight loop, or a single IP opens ten thousand sockets and holds them idle, or a legitimate reconnect storm after a deploy lands forty thousand TLS handshakes on three nodes in two seconds. In each case the expensive part is not the messages — it is the handshakes: TLS negotiation, HTTP parsing, authentication, session loading and registry updates, all before a single application message flows. Message rate limits, the kind covered in rate limiting WebSocket messages per client, do nothing here because the connections never get that far. Handshakes need their own limits, enforced as early and as cheaply as possible.

Root cause #

A WebSocket connection is cheap to hold and expensive to establish. Holding an idle socket costs a few kilobytes and occasional heartbeats. Establishing one costs a TCP handshake, a TLS handshake (asymmetric cryptography, the most CPU-intensive step), an HTTP upgrade request, and whatever your server does on upgrade — verifying a JWT, loading a session, subscribing to channels, replaying missed messages. A server comfortably holding 100,000 connections may only be able to accept a few thousand per second.

That asymmetry is what floods exploit, deliberately or not. Without limits, one misbehaving client or one IP can consume accept capacity that thousands of legitimate users need, and a synchronized reconnect after an outage can push every node past its accept capacity at once, so handshakes time out, clients retry, and the storm feeds itself.

Handshakes per second after a fleet-wide disconnect Reconnect attempts per second for 30000 dropped clients: a fixed retry delay lands every client in one second, full jitter spreads them across 5 seconds. Handshakes per second after a fleet-wide disconnect 30k clients, 5s window — fixed delay vs full jitter Fixed delay Full jitter 0 10k 20k 30k 5.9k 1s 6.0k 2s 6.0k 3s 6.1k 4s 6.1k 5s
The peak second of a reconnect storm, not the average, is what handshake capacity and limits must be designed around.

Resolution #

Limit in layers, from cheapest to most specific. At the edge (nginx, a load balancer or a CDN), limit the rate of new upgrade requests per client IP and cap concurrent connections per IP — this rejects abuse before TLS handshakes reach your application. In the application, limit per authenticated identity, since many legitimate users can share an IP behind NAT. And globally, cap the handshake rate each node accepts so it degrades gracefully instead of falling over.

# Edge: per-IP handshake rate and concurrent-connection caps (nginx).
limit_req_zone  $binary_remote_addr zone=ws_handshake:20m rate=5r/s;   # new upgrades per IP
limit_conn_zone $binary_remote_addr zone=ws_conns:20m;                 # open sockets per IP

server {
    location /ws/ {
        limit_req  zone=ws_handshake burst=20 nodelay;   # absorb a page with a few tabs
        limit_conn ws_conns 50;                          # generous for NAT, fatal for floods
        limit_req_status 429;
        limit_conn_status 429;
        proxy_pass http://realtime;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
// Application: per-user token bucket in Redis, plus a per-node accept budget.
import type { IncomingMessage } from 'node:http';
import type { Duplex } from 'node:stream';
import { createClient } from 'redis';

const USER_BUCKET_CAPACITY = 10;           // burst of reconnects allowed per user
const USER_REFILL_PER_SEC = 0.5;           // one new connection every 2 s sustained
const NODE_MAX_ACCEPTS_PER_SEC = 2_000;    // what this node can authenticate per second
const redis = createClient(); await redis.connect();

// Atomic token bucket: returns 1 if a token was taken, 0 if empty.
const BUCKET_LUA = `
local cap, rate, now = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3])
local s = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(s[1]) or cap
local ts = tonumber(s[2]) or now
tokens = math.min(cap, tokens + (now - ts) * rate)
local ok = 0
if tokens >= 1 then tokens = tokens - 1; ok = 1 end
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', KEYS[1], math.ceil(cap / rate) + 1)
return ok`;

let acceptsThisSecond = 0;
setInterval(() => { acceptsThisSecond = 0; }, 1_000).unref();

function reject(socket: Duplex, status: number, retryAfterSec: number) {
  socket.write(`HTTP/1.1 ${status} Too Many Requests\r\nRetry-After: ${retryAfterSec}\r\nConnection: close\r\n\r\n`);
  socket.destroy();
}

export async function admitHandshake(req: IncomingMessage, socket: Duplex, userId: string): Promise<boolean> {
  // 1. Node-level budget: shed load before doing any expensive work.
  if (++acceptsThisSecond > NODE_MAX_ACCEPTS_PER_SEC) { reject(socket, 503, 2); return false; }
  // 2. Per-user bucket: one client's loop cannot starve everyone else.
  const ok = await redis.eval(BUCKET_LUA, {
    keys: [`wsrl:${userId}`],
    arguments: [String(USER_BUCKET_CAPACITY), String(USER_REFILL_PER_SEC), String(Date.now() / 1000)],
  });
  if (ok !== 1) { reject(socket, 429, Math.ceil(1 / USER_REFILL_PER_SEC)); return false; }
  return true;
}

Order matters: check the cheap node budget before the Redis call, and do the per-user check after authenticating the token but before loading sessions or replaying history. For the per-user key you need an identity, which means a validated token — see validating JWT on the WebSocket upgrade. For unauthenticated endpoints, key on the client IP from a trusted X-Forwarded-For instead.

Rejected handshakes should tell clients to back off. Browsers cannot read the status code of a failed upgrade (they see 1006), so the Retry-After header mainly helps non-browser clients; for browsers, the defence is client-side jittered backoff, which your limits then enforce for clients that do not implement it. Where you accept the upgrade and then decide to shed load, close with 1013 Try Again Later, which clients can read and should honour.

Handshake limits by layer Handshake limits applied at the CDN edge, the load balancer, a per-node accept budget, a per-identity token bucket in Redis, and a concurrent-connection cap per user in the connection registry. Handshake limits by layer CDN / edge per-IP rate + connection caps; absorbs floods before TLS reaches you cheapest Load balancer connection caps per target; slow-start for new nodes cheap Node accept budget max handshakes/s this process can authenticate 503 / 1013 Per-identity bucket Redis token bucket keyed by user or API key 429 Connection registry max concurrent sockets per user close 4029 Each layer catches what the one above cannot see: the edge sees IPs, only the app sees users
Reject early and cheaply; reject precisely where identity is known.

Edge cases #

NAT and shared IPs. Offices, universities and mobile carriers put thousands of users behind one address. Per-IP limits must be generous (tens of connections, a few new per second with a burst) and serve as flood protection only; fairness between real users comes from the per-identity limit.

Tabs and multiple devices. One user with eight tabs legitimately opens eight connections and reconnects all of them together. Size the per-user bucket’s burst for that, or reduce the need with one WebSocket shared across browser tabs.

Deploy storms are legitimate. A graceful deploy that drains connections produces a reconnect wave you caused. Limits should slow it down, not reject it outright — stagger drains as in WebSocket graceful shutdown in Node.js so the wave stays under the node budget.

Verification #

Test each layer separately with a load tool that opens connections rather than sending messages:

# 200 upgrade attempts from one IP in ~2 s: expect most beyond the burst to get 429.
for i in $(seq 1 200); do
  curl -s -o /dev/null -w '%{http_code}\n' --http1.1 https://rt.example.com/ws/ \
    -H 'Connection: Upgrade' -H 'Upgrade: websocket' -H 'Sec-WebSocket-Version: 13' \
    -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' --max-time 1 &
done | sort | uniq -c

Then run a reconnect-storm test: hold your target connection count, restart a node, and confirm that the remaining nodes’ accept rate plateaus at the budget while every client eventually reconnects. In production, export handshake rejections by layer and reason; a spike in per-user rejections from one identity is a looping client, a spike in node-budget rejections is capacity.

A looping client, contained A bad client release starts reconnecting in a loop; after a burst of ten allowed handshakes the per-user bucket returns 429s and admits one connection every two seconds until a fix is deployed, leaving the rest of the fleet unaffected. A looping client, contained user limited, fleet unaffected bad release: reconnect loop (0 s) burst of 10 allowed (0.5 s) 429s begin (1 s) 1 accept every 2 s (2 s) fix deployed (15 s) Without the bucket, the same loop costs a TLS handshake and a JWT check every few milliseconds
The per-identity bucket turns one broken client into one throttled client.

Operational checklist #

FAQ #

Should I rate limit WebSocket connections or messages? #

Both, separately. Handshake limits protect accept capacity — TLS, authentication, session setup — which is the scarcest resource. Message limits protect processing once connections exist. Neither substitutes for the other.

What status code should a rejected WebSocket handshake use? #

429 Too Many Requests with Retry-After for rate limits, 503 Service Unavailable for node overload. Browsers only see a failed connection, so also rely on client backoff; if you accept the upgrade first, close with 1013 instead.

Can Cloudflare or a CDN rate limit WebSocket upgrades? #

Yes. CDN and WAF rate-limiting rules apply to the upgrade request like any other HTTP request, and they are the cheapest place to stop floods. They cannot see your application identity, so keep per-user limits in the app.

How do I stop my own clients from causing reconnect storms? #

Use exponential backoff with full jitter, treat 1013 and 429-style failures as signals to slow down, and stagger server-initiated disconnects. Limits are the safety net, not the primary control.

Back to Security & TLS Configuration.