Capacity planning for WebSocket fleets #

“How many servers do we need for 500,000 concurrent users?” For HTTP services, the answer comes from requests per second and CPU per request. For WebSocket services, that approach produces a number that looks fine on paper and fails on the first bad day, because a real-time fleet has three independent capacity limits — how many connections it can hold, how many messages it can deliver, and how many connections it can accept at once after an outage — and the fleet must be sized for the worst of the three, plus room to lose a node or a zone. This page turns measured per-node numbers into a fleet size you can defend and re-run whenever the workload changes.

Root cause #

WebSocket load has three dimensions that scale independently.

Held connections consume memory (heap, native buffers, kernel socket buffers) and a little CPU for heartbeats, whether or not any messages flow. A fleet of idle dashboards is memory-bound.

Deliveries — each message multiplied by the number of recipients it fans out to — consume CPU for serialization and sending. A chat product with large rooms, or a market-data feed, is delivery-bound; its capacity is measured in deliveries per second, not in published messages per second.

Accepts — new connections per second — consume CPU for TLS handshakes, authentication and session setup, and hit kernel accept queues. In steady state accept load is modest, but after a deploy, a node failure or a regional failover, a large fraction of the fleet reconnects within a minute. Accept capacity during that storm, not steady-state load, often decides how much headroom a fleet needs.

Planning on only one dimension — usually connections — is how fleets end up fine on Tuesday and collapsing during Wednesday’s deploy.

Memory per node against a 3 GB budget Estimated heap for 5,000 to 100,000 concurrent sockets at 30 kilobytes per socket plus 10 kilobytes of application state, against a 3072 megabyte pod limit. Memory per node against a 3 GB budget 30 KB socket + 10 KB app state each — dashed line is a 3072 MB pod limit Socket + buffers App state 586 MB 20k 781 MB 1.2k MB 391 MB 40k 1.6k MB 1.8k MB 586 MB 60k 2.3k MB 2.3k MB 781 MB 80k 3.1k MB 2.9k MB 977 MB 100k 3.9k MB 3072 MB pod limit
Held connections set the memory floor: at about 40 KB per connection, a 3 GB budget holds roughly 75,000 connections before headroom.

Resolution #

Measure three per-node capacities in load tests — max connections at your memory budget, max deliveries per second at your latency target, and max accepts per second without queue overflow — then compute the node count each dimension requires at peak, take the largest, and add failure headroom. Encode the model in code so it is reviewable and repeatable.

interface NodeCapacity {
  maxConnections: number;          // held at memory budget, from density tests
  maxDeliveriesPerSec: number;     // at p99 latency target, from fan-out tests
  maxAcceptsPerSec: number;        // handshakes/s without accept-queue overflow or auth backlog
}

interface Workload {
  peakConnections: number;
  peakPublishesPerSec: number;
  avgFanout: number;               // recipients per publish (room size effect)
  stormReconnectWindowSec: number; // how long clients take to reconnect after a mass disconnect
  stormFraction: number;           // share of all connections reconnecting at once (1 = full fleet)
}

interface Policy {
  targetUtilisation: number;       // e.g. 0.7: keep 30% slack in steady state
  survive: 'node' | 'zone';        // failure to absorb without breaching limits
  zones: number;
}

export function planFleet(node: NodeCapacity, w: Workload, p: Policy) {
  const deliveriesPerSec = w.peakPublishesPerSec * w.avgFanout;
  const byConnections = w.peakConnections / (node.maxConnections * p.targetUtilisation);
  const byDeliveries = deliveriesPerSec / (node.maxDeliveriesPerSec * p.targetUtilisation);
  // Storm: reconnecting clients spread over the window must fit the fleet's accept capacity.
  const stormAcceptsPerSec = (w.peakConnections * w.stormFraction) / w.stormReconnectWindowSec;
  const byAccepts = stormAcceptsPerSec / node.maxAcceptsPerSec;

  let nodes = Math.ceil(Math.max(byConnections, byDeliveries, byAccepts));
  // Headroom: after losing a node or a zone, the survivors must still fit the load.
  if (p.survive === 'node') nodes += 1;
  else nodes = Math.ceil(nodes * p.zones / (p.zones - 1));

  const binding = [['connections', byConnections], ['deliveries', byDeliveries], ['accepts', byAccepts]]
    .sort((a, b) => (b[1] as number) - (a[1] as number))[0][0];
  return { nodes, binding, byConnections: +byConnections.toFixed(1), byDeliveries: +byDeliveries.toFixed(1), byAccepts: +byAccepts.toFixed(1), stormAcceptsPerSec };
}

console.table(planFleet(
  { maxConnections: 60_000, maxDeliveriesPerSec: 200_000, maxAcceptsPerSec: 1_500 },
  { peakConnections: 500_000, peakPublishesPerSec: 4_000, avgFanout: 25, stormReconnectWindowSec: 60, stormFraction: 0.34 },
  { targetUtilisation: 0.7, survive: 'zone', zones: 3 },
));
// → which dimension binds, and how many nodes survive losing a zone

The inputs come from measurements, not vendor claims. Per-node connections and memory come from the density test in measuring memory per WebSocket connection; deliveries and latency from fan-out tests like those in benchmarking Node.js WebSocket servers; accepts from a storm test with production-like TLS and authentication. The storm fraction reflects your worst credible event: losing one zone of three disconnects about a third of the fleet, and a full deploy without staggering disconnects everything.

Which dimension binds, by product type Idle dashboards and small-room chat are usually bound by held connections; live events with huge rooms and market-data feeds are bound by deliveries; mobile-heavy consumer apps with frequent reconnects are often bound by accept capacity during storms. Which dimension binds, by product type Connections Deliveries Accepts (storm) Idle dashboards binds low moderate Chat, small rooms binds moderate moderate Live events, huge rooms moderate binds high Market data feeds low binds moderate Mobile-heavy consumer app moderate moderate binds Compute all three every time — the binding dimension changes as the product changes
Different products hit different walls first.

Headroom and elasticity #

Two kinds of headroom are easy to conflate. Steady-state slack (the targetUtilisation above) absorbs normal fluctuation and keeps latency low; running real-time nodes above about 70–80% of their measured limits leaves no room for bursts. Failure headroom absorbs losing capacity: a node crash, a zone outage, a deploy that takes a fraction of nodes out of service at a time. The zone rule — size so that the surviving zones carry the full load — is the usual standard for anything user-facing.

Autoscaling helps with slow daily curves but not with storms: new nodes take minutes to become ready, while a reconnect storm peaks in seconds. Size accept capacity for storms with standing headroom, spread storms with client jitter and staggered drains, and use autoscaling for the diurnal curve, as in autoscaling WebSockets on Kubernetes with KEDA. Remember that scale-out also needs rebalancing before new nodes actually carry load.

Edge cases #

Shared dependencies. The fleet is only as big as its smallest shared component. Redis pub/sub throughput, database snapshot capacity during resyncs, the auth service during storms and the proxy tier’s own limits all need the same three-dimensional check.

Fan-out growth. Room sizes grow as products succeed, and deliveries grow with room size. Track average fan-out as a first-class metric; a feature that doubles average room size doubles delivery load without adding a single user.

Heterogeneous clients. Mobile clients reconnect far more often than desktop ones. If the mix shifts toward mobile, accept load rises even at constant concurrency.

Verification #

Validate the plan by load-testing the planned fleet shape in staging at peak load, then removing capacity: kill one node, then a zone’s worth of nodes, and confirm that latency stays inside the target and that reconnect storms complete inside the planned window. Re-run the model monthly with current production metrics:

# Inputs for the model, from production (last 30 days, peak hour).
max_over_time(sum(ws_connections_open)[30d:5m])                             # peakConnections
max_over_time(sum(rate(ws_messages_published_total[5m]))[30d:5m])           # peakPublishesPerSec
sum(rate(ws_messages_delivered_total[1h])) / sum(rate(ws_messages_published_total[1h]))   # avgFanout

When a dimension’s utilisation at peak crosses the target, the plan says which resource to add — more nodes for connections or deliveries, more accept headroom or better storm spreading for accepts.

From measurements to a fleet size Capacity planning proceeds from per-node measurements and workload measurements to a per-dimension node count, takes the maximum with failure headroom added, and validates the result with a staging test and a monthly refresh. From measurements to a fleet size Measure per node max connections, deliveries/s at p99 target, accepts/s load tests Measure workload peak connections, publishes/s, fan-out, storm fraction production Compute per dimension nodes needed for each, at target utilisation model Take the max, add headroom survive a node or a zone plan Validate and re-run staging test with failures; monthly refresh loop The plan is a function of measurements — re-run it whenever they change
A repeatable pipeline beats a one-off spreadsheet.

Operational checklist #

FAQ #

How many WebSocket connections can one server handle? #

It depends on memory per connection and on what the connections do. Well-tuned Node.js servers commonly hold tens of thousands to over a hundred thousand mostly idle connections per node; heavy fan-out or large per-connection state lowers that. Measure your own per-connection memory and delivery capacity.

Why isn’t connection count enough for capacity planning? #

Because delivery load (messages × recipients) and reconnect storms can exhaust CPU and accept capacity long before memory runs out. Large rooms, feeds and mobile-heavy audiences are often limited by those instead.

How much headroom should a WebSocket fleet have? #

Enough to run at roughly 70% of measured limits in steady state, plus enough nodes that losing a zone (or at least a node) leaves the survivors within their limits. Storm accept capacity usually needs standing headroom, since autoscaling is too slow for it.

Can autoscaling replace capacity planning? #

Autoscaling handles gradual daily curves, but new nodes take minutes to start and existing connections do not move to them without rebalancing. Storms and failures need capacity that already exists.

Back to Load Testing & Capacity Planning.