Load Testing & Capacity Planning #

Real-time systems fail under load in ways HTTP services do not. An HTTP service under stress slows down request by request; a WebSocket fleet under stress holds tens of thousands of connections that all react to the same event at once. A broadcast to a large room turns one message into fifty thousand sends. A deploy disconnects a third of the fleet and they all come back within a minute. A slow node’s clients reconnect to its neighbours, which slow down in turn. None of that shows up in a test that opens connections as fast as possible and measures handshakes per second — which is what most “WebSocket load tests” do.

This area covers testing and sizing real-time infrastructure the way it is actually used: connections ramped over time and held, mostly idle; messages fanned out to realistic room sizes; latency measured from publish to delivery; and failure scenarios — restarts, zone loss, reconnect storms — rehearsed deliberately. It then turns those measurements into a fleet size with headroom for the day things go wrong. It builds on the per-node limits in Connection Limits & OS Tuning and feeds the scaling decisions in Horizontal Scaling on Kubernetes.

Prerequisites #

Load tests are only as meaningful as the environment they run against and the behaviour they simulate:

• A staging environment with production-shaped infrastructure: the same node sizes, proxies, load balancers, pub/sub layer and OS tuning. A test that bypasses the proxy tier misses its limits, which are often reached first.
• Client behaviour you can reproduce: the real reconnect policy (backoff with jitter), heartbeat interval and resume logic, so virtual users behave like real ones during storms — see Auto-Reconnection Strategies.
• Server-side observability: connections, deliveries, event-loop lag, heap and RSS, close codes and handshake results, exported as metrics you can watch during a test, following WebSocket Observability & Monitoring.
• Load generators with raised descriptor limits and enough CPU, running on separate machines close to the target.

Core implementation: a scenario suite #

A useful suite is not one big test but a small set of scenarios, each designed to find one limit. The shared building block is a virtual user that behaves like a real client and measures what users feel — the delay between a message being published and it arriving.

// scenarios.js — k6 scenario suite (run subsets with --env SCENARIO=...)
import { WebSocket } from 'k6/websockets';
import { Trend, Counter } from 'k6/metrics';

const delivery = new Trend('ws_delivery_latency', true);
const closesAbnormal = new Counter('ws_closes_abnormal');
const URL = __ENV.WS_URL;

// A realistic client: jittered reconnect, room membership, occasional sends, latency on receive.
function client(room, sendEveryMs, lifetimeMs) {
  let attempt = 0;
  const open = () => {
    const ws = new WebSocket(`${URL}?room=${room}`);
    let timer;
    ws.onopen = () => {
      attempt = 0;
      if (sendEveryMs) timer = setInterval(() => ws.send(JSON.stringify({ type: 'chat.send', room, sentAt: Date.now(), vu: __VU })), sendEveryMs);
      setTimeout(() => ws.close(1000), lifetimeMs);
    };
    ws.onmessage = (e) => {
      const m = JSON.parse(e.data);
      if (m.sentAt && m.vu !== __VU) delivery.add(Date.now() - m.sentAt);
    };
    ws.onclose = (e) => {
      clearInterval(timer);
      if (e.code === 1000) return;
      closesAbnormal.add(1);
      const cap = Math.min(30000, 500 * 2 ** attempt++);
      setTimeout(open, Math.random() * cap);               // production-like backoff
    };
  };
  open();
}

const SCENARIOS = {
  // 1. Density: many idle connections, held — finds memory and descriptor limits.
  density: { executor: 'ramping-vus', stages: [{ duration: '10m', target: 40000 }, { duration: '10m', target: 40000 }], exec: 'idle' },
  // 2. Fan-out: fewer clients in big rooms, steady publishing — finds delivery limits.
  fanout: { executor: 'constant-vus', vus: 5000, duration: '15m', exec: 'bigRooms' },
  // 3. Soak with storm: realistic mix held for an hour; restart nodes during it by hand or script.
  storm: { executor: 'ramping-vus', stages: [{ duration: '10m', target: 20000 }, { duration: '50m', target: 20000 }], exec: 'mixed' },
};

export const options = {
  scenarios: { [__ENV.SCENARIO]: SCENARIOS[__ENV.SCENARIO] },
  thresholds: {
    ws_delivery_latency: ['p(95)<250', 'p(99)<1000'],
    ws_connecting: ['p(95)<1500'],
  },
};

export function idle() { client(`idle-${__VU % 1000}`, 0, 20 * 60 * 1000); }
export function bigRooms() { client(`big-${__VU % 5}`, 10000, 15 * 60 * 1000); }       // 1,000 per room
export function mixed() { client(`room-${__VU % 400}`, 45000, 60 * 60 * 1000); }       // ~50 per room

Each scenario answers one question. Density tells you how many connections a node holds within its memory budget and whether memory is linear in connections. Fan-out tells you how many deliveries per second a node sustains at your latency target. Storm tells you whether the fleet recovers from mass disconnects within an acceptable window, and whether admission control and client backoff keep the recovery orderly. The per-tool details are in load testing WebSockets with k6 and load testing WebSockets with Artillery.

Reading results: locating the bottleneck #

A failed threshold says that the system struggled, not where. Reading client-side and server-side metrics together locates the limit quickly, because each resource fails with a characteristic signature.

Memory-bound nodes show heap or RSS approaching the container limit while CPU and event-loop lag are fine, with latency degrading only when garbage collection starts running constantly. Fix: fewer connections per node, less per-connection state, or smaller compression windows.

Delivery-bound nodes show event-loop lag climbing with publish rate, CPU pinned on the main thread, and delivery latency rising sharply past a knee. Fix: serialize once per broadcast, coalesce superseding updates, shard hot rooms, or add nodes.

Accept-bound fleets show handshake latency (ws_connecting) rising and kernel ListenOverflows increasing during storms, while steady-state metrics look healthy. Fix: raise accept queues, add handshake budgets and client jitter, keep standing headroom.

Dependency-bound systems show your nodes idle while Redis, the database or the auth service saturates. The fleet cannot be larger than its smallest shared component; test those components under the same peaks.

Testing the tiers around the WebSocket nodes #

A WebSocket node rarely fails alone; the layers around it have their own limits, and a test that only watches the nodes misses them. Plan explicit checks for each.

The proxy and load-balancer tier holds two sockets per client, consumes ephemeral ports toward each backend, and tracks two conntrack entries per connection. Under a density test, watch proxy memory, descriptor counts and upstream connection counts per backend address; a proxy that stalls near 28,000 upstream sockets per backend is exhausting its port range, as explained in ephemeral port exhaustion on WebSocket proxies. During storm tests, watch the proxy’s accept queues and TLS CPU as closely as the nodes’.

The pub/sub layer carries every cross-node message. Its load is published messages multiplied by the number of nodes with subscribers, not by recipients, so it often looks fine until rooms spread across many nodes. Measure its CPU, network and output-buffer usage during the fan-out scenario, and confirm that a broker restart during the soak test is followed by clean resubscription and resume.

The authentication and session services are hit on every handshake. In steady state that is modest; in a storm, every reconnecting client validates a token or loads a session within the same minute. Include them in the storm scenario, and give them their own rate limits and caches so a reconnect wave does not become an authentication outage.

The data tier absorbs resyncs: clients that cannot resume from their sequence number request snapshots. A storm with a cold replay buffer can turn into thousands of snapshot queries per second. Measure snapshot rates during storms and size replay buffers so most clients resume rather than resync. A replay buffer that covers two or three minutes of traffic per stream is usually enough to turn a storm’s resyncs into cheap replays.

From results to a fleet size #

With per-node limits measured, sizing is arithmetic — but it must be done per dimension. Divide peak connections by per-node connection capacity, peak deliveries (publishes × average fan-out) by per-node delivery capacity, and storm accepts (connections disconnected × fraction, spread over the reconnect window) by per-node accept capacity. Apply a steady-state utilisation target, take the largest of the three, and add failure headroom so that losing a node — or a zone — leaves the survivors within their limits. The full model, with code, is in capacity planning for WebSocket fleets.

Two habits keep the plan honest. First, record which dimension binds. A chat product today may be connection-bound, and the same product after launching large community rooms becomes delivery-bound; the plan should say so, so the next investment goes to the right place. Second, re-run the model from production metrics on a schedule, rather than from launch-day estimates, because fan-out and reconnect behaviour drift as products and audiences change.

Configuration reference #

Edge cases & gotchas #

The generator is the bottleneck. A single load generator process holding 50,000 connections may be the thing that saturates. Watch its CPU and event-loop lag, and add generators until adding more does not change the result.

Same-host testing. Running the generator on the server’s machine makes them compete for CPU and memory and hides network effects. Always test across the network, through the real proxy tier.

Unrealistic clients. Virtual users that reconnect instantly, or never reconnect, or send every second when real users send every minute, produce results that describe a different product. Derive behaviour from production telemetry.

Short tests. Memory leaks, heap fragmentation, timer accumulation and slow presence growth need hours to show. Run a soak test before every major release, not only short ramps.

Clock skew in latency. Publish-to-receive latency computed across machines includes clock differences. Keep senders and receivers on the same generator, or have the server stamp messages.

Testing through a CDN. Load testing through a CDN or DDoS provider can trigger its protection and measure the edge rather than your fleet. Coordinate with the provider or test the origin directly for capacity work.

Verification #

Treat the test suite itself as something to verify. Prove that each scenario can fail: inject a synchronous delay into the broadcast path and confirm the fan-out thresholds fail; cut container memory and confirm the density test fails; disable client jitter and confirm the storm scenario’s handshake rejections spike. A suite that never fails provides false confidence.

Then close the loop with production. After each capacity change, compare predicted and observed peak utilisation per dimension:

# Observed utilisation per dimension at peak, as a fraction of measured per-node limits.
max_over_time((sum(ws_connections_open) / (count(up{job="realtime"}) * 60000))[7d:5m])
max_over_time((sum(rate(ws_messages_delivered_total[1m])) / (count(up{job="realtime"}) * 200000))[7d:5m])
max_over_time((sum(rate(ws_upgrades_total[1m])) / (count(up{job="realtime"}) * 1500))[7d:5m])

If observed utilisation differs from the plan by more than about 20%, one of the model’s inputs is wrong — usually fan-out or per-connection memory — and should be re-measured.

Guides in this area #

• Load Testing WebSockets with k6 — scripted virtual users, publish-to-receive latency metrics, thresholds and storm scenarios.
• Load Testing WebSockets with Artillery — arrival phases, long-lived flows, latency processors and distributed runs in YAML.
• Benchmarking Node.js WebSocket Servers — comparing ws, uWebSockets.js and Socket.IO on density, fan-out and tail latency fairly.
• Capacity Planning for WebSocket Fleets — a three-dimensional sizing model with failure headroom, encoded as code.

FAQ #

How do you load test WebSockets? #

Ramp connections gradually, hold them for a sustained period with realistic idle and send behaviour, measure publish-to-receive latency in the receiving clients, and run failure scenarios such as server restarts. Tools like k6 and Artillery support all of this; the realism comes from the scenario design.

How many concurrent WebSocket connections should I test? #

At least your expected peak plus your failure headroom — typically 1.5 times peak — and a storm scenario where a zone’s worth of connections reconnects at once. Test beyond that to find the actual limit, so you know your margin.

What latency should a WebSocket system target? #

It depends on the product: chat and collaboration usually target p95 delivery under a few hundred milliseconds; trading and gaming much lower. Define the target from user experience first, then test against it.

How often should load tests run? #

A reduced-scale regression run nightly or per release catches performance regressions early; full-scale scenario and soak tests before major launches, infrastructure changes and seasonal peaks.

Can I load test in production? #

Carefully, and only with synthetic traffic that is clearly isolated — separate rooms, rate-limited generators and a kill switch. Production tests reveal real infrastructure limits that staging misses, but staging should always find the obvious problems first.

What’s the difference between a load test and a soak test? #

A load test pushes toward peak concurrency and throughput to find limits; a soak test holds realistic load for hours to find slow problems — memory leaks, growing presence sets, timer accumulation, log volume — that never show up in a twenty-minute run.

Related #

• Connection Limits & OS Tuning — raising the limits load tests find.
• Horizontal Scaling on Kubernetes — acting on the capacity plan.
• Multi-Region & Edge Delivery — the largest failure scenarios to plan for.
• Serverless & Managed WebSockets — load testing platforms you do not operate.

Back to Scaling Real-Time Infrastructure.