WebSocket Scalability: Horizontal Scaling, Sticky Sessions, and Redis Pub/Sub

A single WebSocket server is easy: clients connect, you maintain state in memory, and you broadcast to connected clients. Horizontal scaling breaks this — when you add a second server, a message sent to server A can't reach clients connected to server B.

This guide covers the patterns that make WebSocket servers horizontally scalable without sacrificing real-time delivery.

---

The Core Problem

Client A → Server 1 (connected)
Client B → Server 2 (connected)

Client A sends message "Hello to everyone"
Server 1 receives it
Server 1 can only reach: Client A (and other clients on Server 1)
Server 1 cannot reach: Client B (on Server 2)

Result: Client B never receives the message

Two solutions:

1. Sticky sessions: Route each client to the same server, always (partial solution)
2. Redis Pub/Sub: Broadcast messages between servers (full solution)

---

Sticky Sessions with Nginx

Sticky sessions ensure a client always routes to the same server. This helps for connection state but doesn't solve cross-server broadcasting:

# nginx.conf — sticky sessions for WebSocket
upstream websocket_servers {
  ip_hash;  # Route same IP to same server (sticky)
  # Or use consistent hashing:
  # hash $http_x_user_id consistent;  # If client sends user ID in header

  server ws-server-1:3000;
  server ws-server-2:3000;
  server ws-server-3:3000;

  keepalive 1000;  # Keep upstream connections alive
}

server {
  listen 443 ssl;

  location /ws {
    proxy_pass http://websocket_servers;

    # WebSocket upgrade headers
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    # Keep WebSocket connections alive
    proxy_read_timeout 3600s;  # 1 hour timeout (extend for long-lived connections)
    proxy_send_timeout 3600s;
  }
}

Sticky sessions alone aren't sufficient — if a server restarts, clients lose their connection and must reconnect. They may reconnect to a different server, losing any in-memory state.

---

Redis Pub/Sub: The Actual Solution

Redis Pub/Sub lets any server publish to a channel that all servers subscribe to. When Client A sends a message, Server 1 publishes to Redis; all servers (including Server 2) receive it and deliver to their connected clients.

// lib/websocket/wsServer.ts
import { WebSocketServer, WebSocket } from 'ws';
import Redis from 'ioredis';

// Separate Redis connections for pub and sub
const publisher = new Redis(process.env.REDIS_URL!);
const subscriber = new Redis(process.env.REDIS_URL!);

// Track connected clients per server instance
const clients = new Map<string, WebSocket>();  // userId → WebSocket

const wss = new WebSocketServer({ port: 3001 });

// Subscribe to messages from other server instances
subscriber.subscribe('ws:broadcast', 'ws:dm', (err) => {
  if (err) throw err;
  console.log('Subscribed to Redis channels');
});

subscriber.on('message', (channel: string, message: string) => {
  const payload = JSON.parse(message);

  if (channel === 'ws:broadcast') {
    // Deliver to all clients connected to THIS instance
    for (const [, ws] of clients) {
      if (ws.readyState === WebSocket.OPEN) {
        ws.send(JSON.stringify(payload));
      }
    }
  } else if (channel === 'ws:dm') {
    // Deliver to specific user if connected to THIS instance
    const targetWs = clients.get(payload.targetUserId);
    if (targetWs?.readyState === WebSocket.OPEN) {
      targetWs.send(JSON.stringify(payload.message));
    }
  }
});

wss.on('connection', (ws: WebSocket, req) => {
  const userId = extractUserId(req);  // From JWT in query param or cookie
  if (!userId) {
    ws.close(1008, 'Unauthorized');
    return;
  }

  // Register client
  clients.set(userId, ws);
  console.log(`Client connected: ${userId} (${clients.size} total on this instance)`);

  ws.on('message', async (data) => {
    const message = JSON.parse(data.toString());
    await handleClientMessage(userId, message);
  });

  ws.on('close', () => {
    clients.delete(userId);
    console.log(`Client disconnected: ${userId}`);
  });

  ws.on('error', (err) => {
    console.error(`WebSocket error for ${userId}:`, err);
    clients.delete(userId);
  });

  // Send initial state
  ws.send(JSON.stringify({ type: 'connected', userId, timestamp: Date.now() }));
});

// Broadcast to ALL connected clients (across all server instances)
export async function broadcastToAll(message: unknown): Promise<void> {
  await publisher.publish('ws:broadcast', JSON.stringify(message));
}

// Send to specific user (wherever they're connected)
export async function sendToUser(targetUserId: string, message: unknown): Promise<void> {
  await publisher.publish('ws:dm', JSON.stringify({
    targetUserId,
    message,
  }));
}

async function handleClientMessage(userId: string, message: any): Promise<void> {
  switch (message.type) {
    case 'chat:send':
      // Save to DB
      await db.messages.create({
        roomId: message.roomId,
        userId,
        content: message.content,
      });
      // Broadcast to all instances (including this one)
      await broadcastToAll({
        type: 'chat:message',
        roomId: message.roomId,
        userId,
        content: message.content,
        timestamp: Date.now(),
      });
      break;

    case 'typing:start':
      await broadcastToAll({
        type: 'typing:indicator',
        roomId: message.roomId,
        userId,
        isTyping: true,
      });
      break;
  }
}

---

Room-Based Subscriptions

For products with rooms/channels (chat apps, collaborative tools), use per-room Redis channels:

// lib/websocket/roomManager.ts

// Track room memberships per server instance
const roomClients = new Map<string, Set<string>>();  // roomId → Set<userId>

export function joinRoom(userId: string, roomId: string) {
  if (!roomClients.has(roomId)) {
    roomClients.set(roomId, new Set());
    // Subscribe to room channel if first member
    subscriber.subscribe(`room:${roomId}`);
  }
  roomClients.get(roomId)!.add(userId);
}

export function leaveRoom(userId: string, roomId: string) {
  roomClients.get(roomId)?.delete(userId);
  if (roomClients.get(roomId)?.size === 0) {
    roomClients.delete(roomId);
    // Unsubscribe from room channel if last member
    subscriber.unsubscribe(`room:${roomId}`);
  }
}

// Broadcast to all clients in a room (across all instances)
export async function broadcastToRoom(roomId: string, message: unknown): Promise<void> {
  await publisher.publish(`room:${roomId}`, JSON.stringify(message));
}

// Handle room messages
subscriber.on('message', (channel: string, data: string) => {
  if (channel.startsWith('room:')) {
    const roomId = channel.replace('room:', '');
    const message = JSON.parse(data);

    // Deliver to all THIS instance's clients in this room
    const roomUsers = roomClients.get(roomId) ?? new Set();
    for (const userId of roomUsers) {
      const ws = clients.get(userId);
      if (ws?.readyState === WebSocket.OPEN) {
        ws.send(JSON.stringify(message));
      }
    }
  }
});

---

Heartbeat and Reconnection

Clients must handle disconnections gracefully. WebSocket connections die silently — no TCP FIN when a mobile network switches or a laptop sleeps.

// client/websocket.ts
class ReconnectingWebSocket {
  private ws: WebSocket | null = null;
  private heartbeatInterval: ReturnType<typeof setInterval> | null = null;
  private reconnectDelay = 1000;
  private maxDelay = 30_000;
  private url: string;
  private handlers: Map<string, ((data: unknown) => void)[]> = new Map();

  constructor(url: string) {
    this.url = url;
    this.connect();
  }

  private connect() {
    const token = getAuthToken();
    this.ws = new WebSocket(`${this.url}?token=${token}`);

    this.ws.onopen = () => {
      console.log('WebSocket connected');
      this.reconnectDelay = 1000;  // Reset backoff on successful connection
      this.startHeartbeat();
    };

    this.ws.onmessage = (event) => {
      const message = JSON.parse(event.data);
      if (message.type === 'pong') return;  // Heartbeat response
      this.emit(message.type, message);
    };

    this.ws.onclose = (event) => {
      this.stopHeartbeat();
      if (!event.wasClean) {
        console.log(`WebSocket closed unexpectedly — reconnecting in ${this.reconnectDelay}ms`);
        setTimeout(() => this.connect(), this.reconnectDelay);
        this.reconnectDelay = Math.min(this.reconnectDelay * 2, this.maxDelay);
      }
    };

    this.ws.onerror = (error) => {
      console.error('WebSocket error:', error);
      // onclose will fire after onerror
    };
  }

  private startHeartbeat() {
    this.heartbeatInterval = setInterval(() => {
      if (this.ws?.readyState === WebSocket.OPEN) {
        this.ws.send(JSON.stringify({ type: 'ping' }));
      }
    }, 30_000);  // Ping every 30 seconds
  }

  private stopHeartbeat() {
    if (this.heartbeatInterval) {
      clearInterval(this.heartbeatInterval);
      this.heartbeatInterval = null;
    }
  }

  send(message: unknown) {
    if (this.ws?.readyState === WebSocket.OPEN) {
      this.ws.send(JSON.stringify(message));
    }
  }

  on(type: string, handler: (data: unknown) => void) {
    const handlers = this.handlers.get(type) ?? [];
    handlers.push(handler);
    this.handlers.set(type, handlers);
  }

  private emit(type: string, data: unknown) {
    this.handlers.get(type)?.forEach(h => h(data));
  }
}

---

Server-Sent Events: When WebSocket Is Overkill

For one-directional streaming (server → client), Server-Sent Events (SSE) are simpler than WebSocket:

// api/stream.ts — SSE endpoint
app.get('/api/events', async (request, reply) => {
  reply.raw.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    'Connection': 'keep-alive',
    'X-Accel-Buffering': 'no',  // Disable Nginx buffering
  });

  const userId = request.user.id;

  // Subscribe to user's events via Redis
  const sub = new Redis(process.env.REDIS_URL!);
  await sub.subscribe(`user:${userId}:events`);

  sub.on('message', (_, data) => {
    // SSE format: "data: ...\n\n"
    reply.raw.write(`data: ${data}\n\n`);
  });

  // Keep-alive ping every 15s (prevents proxy timeouts)
  const keepAlive = setInterval(() => {
    reply.raw.write(': ping\n\n');
  }, 15_000);

  request.raw.on('close', () => {
    clearInterval(keepAlive);
    sub.unsubscribe();
    sub.quit();
  });
});

SSE advantages over WebSocket: simpler to scale (stateless HTTP), automatic browser reconnection, works through HTTP/2 multiplexing. Use SSE for notifications, live feeds, and status updates. Use WebSocket for bidirectional communication (chat, collaborative editing, gaming).

---

Working With Viprasol

We build real-time infrastructure — WebSocket servers with Redis pub/sub for horizontal scaling, reconnection handling, room-based messaging, and SSE for server-push use cases. Real-time features handled correctly are invisible to users; handled poorly, they're the most visible reliability failures.

→ Talk to our team about real-time feature architecture.

---

See Also

• Real-Time Collaboration — CRDTs and Yjs on top of WebSocket
• Redis Use Cases — Redis pub/sub for cross-server messaging
• Event-Driven Architecture — event-driven patterns for real-time systems
• Caching Strategies — caching to reduce WebSocket message volume
• Web Development Services — real-time product development