Profiling CPU in a Node.js WebSocket server #
At peak hours your WebSocket nodes sit at 95% CPU, delivery latency climbs past a second, and the obvious fix — more nodes — only moves the problem, because every node runs the same hot code. Somewhere in the message path something is expensive, and guessing is not working: the team has already switched JSON libraries and tuned garbage collection with no effect. A Node.js WebSocket server runs almost everything on one thread, which makes CPU problems both severe (one slow handler delays every connection on the node) and very tractable, because a profile of that one thread shows exactly where the time goes. This page covers measuring the symptom, capturing a profile safely in production-like conditions, and reading it for the patterns that usually turn out to be responsible.
Root cause #
Node.js executes JavaScript on a single main thread. Every WebSocket message — parsing, validation, authorization, business logic, serialization, and sending to each recipient — runs there, as do heartbeats, timers and new-connection handling. When the total work per second exceeds what one core can do, the event loop falls behind: messages queue, pongs are answered late, and latency rises for every client on the node. The metric that reveals it is event-loop lag (or event-loop utilisation), not overall CPU percentage, which can look moderate on a multi-core machine while the one thread that matters is saturated.
The expensive work in real-time servers clusters in a few places. Fan-out paths that serialize a message once per recipient instead of once per broadcast. Validation or authorization that runs a database or regex-heavy check per message. Logging that stringifies whole payloads. Compression (permessage-deflate) on large or frequent frames. And synchronous work hidden in libraries — crypto, JSON of large objects, deep clones — that looks cheap in a unit test and dominates at 50,000 messages a second.
Resolution #
Work from symptom to cause in three steps: measure event-loop health continuously, capture a CPU profile under realistic load, and attribute handler time by message type so regressions can be traced without a profiler.
import { monitorEventLoopDelay, performance } from 'node:perf_hooks';
import { Histogram } from 'prom-client';
// 1. Event-loop health, exported as metrics (utilisation is the most direct saturation signal).
const lag = monitorEventLoopDelay({ resolution: 10 });
lag.enable();
let lastElu = performance.eventLoopUtilization();
setInterval(() => {
const elu = performance.eventLoopUtilization(lastElu); // fraction of time the loop was busy
lastElu = performance.eventLoopUtilization();
metrics.eventLoopUtilisation.set(elu.utilization);
metrics.eventLoopLagP99.set(lag.percentile(99) / 1e6); // ms
lag.reset();
}, 5_000).unref();
// 3. Handler time per message type — cheap enough to leave on in production.
const handlerSeconds = new Histogram({
name: 'ws_handler_seconds', help: 'Time spent in message handlers', labelNames: ['type'],
buckets: [0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.1],
});
export function timed<T>(type: string, fn: () => T): T {
const end = handlerSeconds.startTimer({ type });
try { return fn(); } finally { end(); }
}
// router: ws.on('message', (raw) => timed(msg.type, () => handle(msg)));
declare const metrics: { eventLoopUtilisation: { set(v: number): void }; eventLoopLagP99: { set(v: number): void } };
For step 2, capture a profile while the server handles realistic load — replay production traffic in staging with a load test, or profile a single production node for a short window. Node’s built-in profiler writes a .cpuprofile file that Chrome DevTools opens as a flame chart:
# Start the server with the sampling profiler (writes CPU.*.cpuprofile on exit).
node --cpu-prof --cpu-prof-dir=./profiles dist/server.js &
PID=$!
# Drive realistic load for 60 s (see the k6 guide), then stop the server cleanly.
k6 run --duration 60s chat-load.js
kill -INT $PID
# Open ./profiles/*.cpuprofile in Chrome DevTools → Performance (or speedscope.app).
# Alternatively, attach to a running process without a restart:
kill -USR1 $PID # enables the inspector; connect Chrome DevTools, record a CPU profile
In the flame chart, look at the widest stacks under your message handlers. The patterns that recur in WebSocket servers are recognisable: JSON.stringify called from inside a loop over sockets (serialize once per broadcast instead, as in WebSocket rooms and channel subscriptions); deflate frames from permessage-deflate on high-rate small messages; schema compilation happening per message instead of once at startup; and logger serialization of entire payloads. For a quicker first look, clinic flame and clinic doctor wrap the same data with guided diagnosis.
Edge cases #
Profiling overhead. The sampling profiler adds modest overhead, acceptable in staging and for short windows on one production node behind a load balancer. Do not leave it running fleet-wide.
Work outside JavaScript. TLS encryption and zlib run partly in native code and in the libuv thread pool; they appear in profiles as native frames or not at all. If the main thread looks idle but CPU is high, check TLS termination and compression, or move TLS to a proxy.
Worker threads. Offloading CPU-heavy work (large snapshot generation, compression, image processing) to worker threads keeps the event loop responsive, but it adds copying costs. Offload only work that is both expensive and independent of the socket objects.
Verification #
Confirm each fix with the same load and the same metrics. Event-loop utilisation and p99 lag should drop at the same message rate; delivery latency from the load test should improve; and the per-type handler histogram should show the change for the affected message types:
# Top message types by total handler time, last 15 minutes.
topk(5, sum by (type) (rate(ws_handler_seconds_sum[15m])))
# Event-loop saturation per pod.
max by (pod) (nodejs_eventloop_utilisation)
Keep the handler histogram and event-loop metrics permanently; a new release that adds 0.3 ms to a hot message type shows up there within minutes, long before users notice.
Operational checklist #
FAQ #
How do I find what’s using CPU in my Node.js WebSocket server? #
Start with event-loop utilisation to confirm the main thread is saturated, use per-message-type handler timing to narrow down the hot path, then capture a CPU profile with --cpu-prof or the inspector under realistic load and read the flame graph.
Why is CPU at 50% but latency terrible? #
On a multi-core machine, one saturated core shows as a fraction of total CPU. Node’s main thread can be at 100% while the machine looks half idle. Event-loop utilisation shows the truth.
Is JSON.stringify slow? #
Not for a single message, but at high rates and when called once per recipient in a broadcast loop, it often dominates. Serialize once and send the same string to every socket.
Should I use worker threads for WebSocket handling? #
Keep sockets on the main thread and offload only heavy, self-contained work. For more total throughput, run more processes (one per core) behind the load balancer rather than threading the socket handling itself.
Related #
- Instrumenting WebSockets with OpenTelemetry — traces alongside CPU metrics.
- Benchmarking Node.js WebSocket Servers — when the library itself is the cost.
- Coalescing High-Frequency WebSocket Updates — reducing work at the source.
- Exporting WebSocket Metrics to Prometheus — where these metrics live.
Back to WebSocket Observability & Monitoring.