Running WebSockets behind a CDN #
You put your site behind a CDN for caching and DDoS protection, and the WebSocket endpoint came along with it. Now connections close every 100 seconds when idle, a slice of clients disconnect every few days when the CDN deploys its own edge software, your server’s logs show the CDN’s addresses instead of your users’, and rate limiting by IP throttles an entire office. CDNs proxy WebSockets well — they terminate TLS near the user, absorb connection floods, and hide your origin — but a CDN edge is another stateful hop with its own timeouts, lifecycle and addressing, and it needs the same deliberate configuration as any proxy in the path.
Root cause #
A CDN proxying a WebSocket holds two connections per client: client-to-edge and edge-to-origin. Every property of that edge becomes a property of your service. Idle timeouts are fixed or plan-dependent and often shorter than your origin’s; Cloudflare, for example, closes proxied WebSockets that exchange no data for around 100 seconds. Connection lifetime is bounded by the edge’s own maintenance: when the CDN deploys new edge software or rebalances traffic, long-lived connections on the affected machines are closed, typically with little warning. Addressing changes: your origin sees the edge’s IP address, so logging, rate limiting and geolocation by IP break unless you read the forwarded client address. And features differ: some CDNs require WebSockets to be enabled explicitly, some only proxy them on specific plans or ports, and caching rules must not apply to upgrade requests.
Resolution #
Configure four things: enable WebSocket proxying and bypass caching for the endpoint; keep a heartbeat below the edge’s idle limit; treat edge-initiated disconnects as routine and reconnect smoothly; and restore the real client address at the origin, trusting only the CDN’s ranges.
import http from 'node:http';
import { WebSocketServer } from 'ws';
import ipaddr from 'ipaddr.js';
// 1. Heartbeat well under the CDN's idle limit (Cloudflare ~100 s): 30 s leaves margin.
const HEARTBEAT_MS = 30_000;
// 2. Only trust client-IP headers when the request came from the CDN's published ranges.
// Refresh this list from the CDN's API on a schedule (e.g. Cloudflare /ips).
let CDN_RANGES: [ipaddr.IPv4 | ipaddr.IPv6, number][] = [];
export function setCdnRanges(cidrs: string[]) { CDN_RANGES = cidrs.map((c) => ipaddr.parseCIDR(c)); }
function fromCdn(remote: string) {
const addr = ipaddr.process(remote);
return CDN_RANGES.some(([range, bits]) => addr.kind() === range.kind() && addr.match(range, bits));
}
export function clientIp(req: http.IncomingMessage): string {
const remote = req.socket.remoteAddress ?? '';
if (!fromCdn(remote)) return remote; // direct hit: never trust headers
return (req.headers['cf-connecting-ip'] as string) // Cloudflare
?? (req.headers['true-client-ip'] as string) // Akamai / Cloudflare Enterprise
?? String(req.headers['x-forwarded-for'] ?? '').split(',')[0].trim()
?? remote;
}
const server = http.createServer();
const wss = new WebSocketServer({ server });
wss.on('connection', (ws, req) => {
(ws as any).clientIp = clientIp(req); // for logs, rate limits, geo
let alive = true;
ws.on('pong', () => { alive = true; });
const t = setInterval(() => {
if (!alive) { ws.terminate(); return; }
alive = false;
ws.ping(); // resets the edge idle timer too
}, HEARTBEAT_MS);
ws.on('close', () => clearInterval(t));
});
server.listen(8080);
The heartbeat interval must sit below the shortest idle timer on the path, which behind a CDN is usually the edge — the arithmetic is in tuning WebSocket idle timeouts across proxies. The IP handling matters for security as well as analytics: rate limits and handshake budgets keyed on the edge’s address would throttle thousands of users together, and trusting a forwarded header from anyone would let attackers spoof their address. Lock the origin down so it only accepts traffic from the CDN — by firewall allowlist, authenticated origin pulls (mTLS between edge and origin), or a tunnel — or the CDN’s protection can be bypassed entirely.
Edge restarts are unavoidable, so the client must treat an unexpected close as routine: reconnect with jitter and resume from the last sequence, exactly as in handling WebSocket disconnects gracefully. Because edge maintenance affects a fraction of connections at a time, the reconnects are spread out — but only if clients use jitter.
Edge cases #
Message size and rate limits. Some CDN plans cap WebSocket message size or bandwidth per connection, or apply WAF inspection to frames. Check limits against your largest messages and your peak per-connection rate, and exempt the WebSocket path from WAF rules that buffer or inspect bodies.
Anycast and connection placement. CDN edges are anycast, so a client’s connection lands at the nearest edge, which then connects to your origin. For multi-region origins, use the CDN’s load balancing or origin steering to connect each edge to the nearest healthy region; see routing clients to the nearest WebSocket region.
Connection limits at the origin. The CDN terminates client connections, but the origin still holds one connection per client from the edge. Size origin descriptors, ports and memory for the full client count; the CDN does not multiplex WebSockets.
Verification #
Measure the edge’s behaviour directly. Open an idle connection through the CDN with heartbeats disabled and time how long it survives — that is your edge idle limit. Re-enable heartbeats and confirm the connection survives indefinitely. Check that the origin logs the real client address, and that a direct request to the origin (bypassing the CDN) is refused:
# 1. Upgrade through the CDN must return 101 and must not be cached.
curl -si --http1.1 https://rt.example.com/ws -H 'Connection: Upgrade' -H 'Upgrade: websocket' \
-H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' --max-time 2 | \
grep -iE '^HTTP|cf-cache-status|x-cache'
# 2. Direct-to-origin attempt must fail (origin locked to CDN ranges).
curl -s -o /dev/null -w '%{http_code}\n' --resolve rt.example.com:443:203.0.113.20 https://rt.example.com/ws --max-time 3
Over time, graph disconnects by close code and time of day: edge maintenance shows up as occasional batches of abnormal closes, which should be absorbed by resume without user-visible errors.
Operational checklist #
FAQ #
Does Cloudflare support WebSockets? #
Yes. Cloudflare proxies WebSocket connections, subject to plan limits and an idle timeout of about 100 seconds without data. Keep a heartbeat under that, and expect occasional disconnects during edge maintenance.
Why do my WebSockets disconnect every 100 seconds behind a CDN? #
The CDN edge closes idle connections. Send a ping or small message every 20–30 seconds so the connection never appears idle.
How do I get the real client IP for WebSockets behind a CDN? #
Read the CDN’s client-IP header (CF-Connecting-IP for Cloudflare, True-Client-IP or X-Forwarded-For elsewhere), but only for requests whose source address is in the CDN’s published ranges — otherwise clients can spoof it.
Does a CDN reduce load on my WebSocket servers? #
It absorbs TLS handshakes, floods and malformed traffic at the edge, but every legitimate client still holds a connection to your origin through it. It improves protection and handshake latency, not origin connection counts.
Related #
- Tuning WebSocket Idle Timeouts Across Proxies — the CDN as one of several timers.
- Handling Region Failover for WebSockets — steering behind the edge.
- Rate Limiting WebSocket Handshakes — per-IP limits that need the real client IP.
- Cloudflare Durable Objects WebSocket Hibernation — running the WebSocket server at the edge itself.
Back to Multi-Region & Edge Delivery.