Istio service mesh WebSocket configuration #

Your WebSocket service worked in plain Kubernetes. After enabling Istio sidecar injection, connections drop after exactly an hour, some clients fail the upgrade with a 426 or a reset, sticky routing for the Socket.IO fallback stops working, and every deploy now cuts connections instantly instead of draining. Istio puts an Envoy proxy next to every pod and on the ingress gateway, and each of those proxies applies HTTP routing, timeouts, load balancing and connection management to your traffic. Most defaults suit request/response services; WebSockets need a handful of deliberate settings, and knowing which Istio resource controls which Envoy behaviour is most of the work.

Root cause #

With Istio, a browser’s WebSocket passes through at least three proxies: the ingress gateway, the client-side sidecar (for mesh-internal callers) and the server-side sidecar in the pod. Istio enables WebSocket upgrades for HTTP routes by default, so the handshake usually works — but four behaviours bite.

Protocol detection. Istio decides whether a port carries HTTP or opaque TCP from the Service port name or appProtocol. If it guesses wrong — a port named ws without a protocol prefix, for instance — it may treat traffic as TCP (losing HTTP routing features) or apply HTTP handling where you did not expect it. Timeouts. Envoy’s stream idle timeout and the mesh’s default idle timeouts close quiet connections, and a timeout set on a VirtualService route applies to the upgraded stream as a whole. Load balancing. Sidecars balance per request by default, so multi-request handshakes (polling fallbacks) scatter across pods without a consistent-hash policy. Draining. During pod shutdown, the sidecar starts draining and may terminate before your application has closed its WebSockets gracefully.

Istio resources and the Envoy behaviour they control The Istio resources that shape WebSocket behaviour: Service port naming decides protocol handling, Gateway and VirtualService control routing and route timeouts, DestinationRule controls consistent hashing and connection pools, sidecar settings tune idle timeouts and draining, and the pod lifecycle governs graceful shutdown. Istio resources and the Envoy behaviour they control Service port name / appProtocol http / http2 / tcp — decides how Envoy parses the port protocol Gateway + VirtualService route match, route timeout, websocket upgrade on HTTP routes routing DestinationRule consistentHash affinity, connection pool idle timeouts balancing Sidecar / EnvoyFilter / annotations stream idle timeout, drain duration, hold-until-proxy-ready proxy tuning Pod lifecycle terminationGracePeriod, preStop, app SIGTERM handling draining Most WebSocket issues in Istio trace back to the first and fourth rows
Five places to configure, each owning a different failure.

Resolution #

Name ports explicitly, route WebSockets on HTTP routes without a finite route timeout, add consistent-hash affinity when handshakes span several requests, raise idle timeouts above your heartbeat interval, and make the sidecar outlive your application’s drain.

apiVersion: v1
kind: Service
metadata: { name: realtime }
spec:
  selector: { app: realtime }
  ports:
    - name: http-ws              # "http" prefix (or appProtocol: http) → Envoy handles the upgrade
      port: 8080
      targetPort: 8080
      appProtocol: http
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata: { name: realtime }
spec:
  hosts: ["rt.example.com"]
  gateways: ["istio-ingress/public"]
  http:
    - match: [{ uri: { prefix: /ws } }]
      route: [{ destination: { host: realtime, port: { number: 8080 } } }]
      timeout: 0s                # no overall deadline for upgraded streams
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata: { name: realtime }
spec:
  host: realtime
  trafficPolicy:
    loadBalancer:
      consistentHash:            # keep a client's handshake requests on one pod
        httpCookie: { name: rt_route, ttl: 0s }
    connectionPool:
      http:
        idleTimeout: 3600s       # upstream connection idle timeout in the sidecar pool
      tcp:
        tcpKeepalive: { time: 300s, interval: 30s }
# Deployment pod template annotations: drain ordering and stream idle timeout.
metadata:
  annotations:
    proxy.istio.io/config: |
      holdApplicationUntilProxyStarts: true
      terminationDrainDuration: 30s        # sidecar keeps serving while the app drains
spec:
  terminationGracePeriodSeconds: 45
  containers:
    - name: realtime
      lifecycle:
        preStop:
          exec: { command: ["sh", "-c", "curl -s -X POST localhost:9102/drain; sleep 25"] }

terminationDrainDuration keeps the sidecar forwarding existing connections for that long after shutdown begins, so the application has time to send 1001 closes in batches — the drain logic in WebSocket graceful shutdown in Node.js. Set it longer than your application’s drain window, and the pod’s grace period longer than both. holdApplicationUntilProxyStarts prevents the opposite race at startup, where the application’s outbound connections (to Redis, for example) fail because the sidecar is not ready yet.

Envoy’s per-stream idle timeout also applies inside the mesh. If your heartbeat interval is long or connections are idle for long periods, raise it (through mesh config or an EnvoyFilter on the HTTP connection manager’s stream_idle_timeout) above the heartbeat interval, and keep the heartbeat running so every hop sees traffic, as in tuning WebSocket idle timeouts across proxies. The Envoy fields underneath are the ones described in Envoy WebSocket proxy configuration.

Pod shutdown with a correctly ordered drain When the pod is terminated, the application stops being ready and closes sockets in waves until twenty-two seconds; the sidecar keeps forwarding until its thirty-second drain duration ends, well before the forty-five-second grace period. Pod shutdown with a correctly ordered drain sidecar still forwarding SIGTERM to app + sidecar (0 s) app: not ready, start 1001 waves (1 s) app: all sockets closed (22 s) sidecar drain ends (30 s) grace period (SIGKILL) (45 s) If the sidecar stopped at 5 s, every remaining socket would end as 1006
The sidecar must outlive the application's drain, and both must beat the grace period.

Edge cases #

mTLS and TCP ports. With strict mutual TLS in the mesh, traffic between sidecars is encrypted regardless of protocol. If you treat the port as TCP to avoid HTTP handling, you lose route-level features (path matching, header-based affinity) but keep mTLS. For browser-facing WebSockets, HTTP handling is almost always what you want.

Consistent hashing and scale changes. Istio’s consistent hashing uses a ring hash; adding pods remaps a fraction of keys. Existing WebSockets stay on their pods, but reconnects may land elsewhere, so session state still belongs in a shared store.

Ambient mode. Istio’s ambient mesh moves layer-4 handling to a per-node proxy and layer-7 handling to optional waypoint proxies. WebSocket upgrades and timeouts then depend on the waypoint’s configuration; verify the same settings apply there if you adopt ambient mode.

Verification #

Check what Envoy actually received, not what you think you applied. istioctl shows the routes, clusters and listeners generated for a pod:

POD=$(kubectl get pod -l app=realtime -o jsonpath='{.items[0].metadata.name}')
# The route for /ws: look for "timeout": "0s" and upgrade configuration.
istioctl proxy-config routes "$POD" -o json | jq '.. | objects | select(.match?.prefix == "/ws")'
# The cluster's load-balancing policy and hash settings.
istioctl proxy-config clusters "$POD" --fqdn realtime.default.svc.cluster.local -o json | jq '.[0].lbPolicy, .[0].ringHashLbConfig'
# Sidecar stats: upgrades and idle-timeout closes.
kubectl exec "$POD" -c istio-proxy -- pilot-agent request GET stats | grep -E 'upgrades_total|idle_timeout'

Then test lifecycle end to end: hold a thousand idle connections through the gateway for longer than your longest suspected timeout, and trigger a rolling restart; clients should see 1001 closes spread across the drain window, not 1006.

Symptom to Istio setting Common Istio WebSocket symptoms mapped to causes and settings: fixed-interval drops to idle timeouts, failed upgrades to port protocol, lost polling sessions to missing consistent hashing, deploy-time 1006 spikes to sidecar drain duration, and startup errors to proxy start ordering. Symptom to Istio setting Likely cause Setting Drops at exactly 1 h / 5 min idle or stream timeout stream_idle_timeout, pool idleTimeout Upgrade fails, 426 / reset port treated as TCP/other port name or appProtocol Polling fallback loses session per-request balancing DestinationRule consistentHash Deploys cause 1006 spikes sidecar exits first terminationDrainDuration Startup Redis errors app before sidecar holdApplicationUntilProxyStarts Always confirm the generated Envoy config with istioctl after a change
Five symptoms, five knobs.

Operational checklist #

FAQ #

Does Istio support WebSockets? #

Yes. HTTP routes allow WebSocket upgrades by default. Problems come from timeouts, protocol detection, load balancing and drain ordering, which need explicit configuration for long-lived connections.

Why do my WebSockets close after an hour in Istio? #

An idle or stream timeout in the sidecar or gateway is expiring — commonly the connection pool’s idle timeout or Envoy’s stream idle timeout. Raise it above your heartbeat interval and keep heartbeats flowing so every proxy sees activity.

How do I get sticky sessions for WebSockets in Istio? #

Use a DestinationRule with consistentHash on a cookie, header or source IP. It affects which pod new connections and handshake requests land on; an established WebSocket stays on its pod regardless.

Why do deploys drop connections abruptly with Istio? #

The sidecar begins shutting down at the same time as your application and may stop forwarding before your graceful close finishes. Set terminationDrainDuration so the sidecar keeps serving for longer than your drain window.

Back to Horizontal Scaling on Kubernetes.