Kubernetes StatefulSets: PVCs, Headless Services, Ordered Scaling, and Operator Patterns

StatefulSets are the Kubernetes primitive for workloads that need stable identity — a database replica that must reconnect to its own data volume, a Kafka broker that must be reachable at a predictable hostname, a Redis cluster member that must rejoin with the same node ID.

Deployments treat pods as interchangeable cattle. StatefulSets treat pods as pets with names, identities, and dedicated storage.

StatefulSet vs Deployment

Deployment:
  - Pods: pod-abc123, pod-def456 (random names)
  - Scale: any order, parallel
  - Storage: shared PVC or ephemeral
  - Network: service load-balances across all pods
  - Use for: stateless apps, APIs, workers

StatefulSet:
  - Pods: redis-0, redis-1, redis-2 (stable ordinal names)
  - Scale: ordered (0, then 1, then 2) by default
  - Storage: each pod gets its own PVC that follows it
  - Network: headless service gives stable DNS per pod
  - Use for: databases, message queues, caches, anything with state

Basic StatefulSet

# kubernetes/statefulsets/redis-cluster.yaml
apiVersion: v1
kind: Service
metadata:
  name: redis-headless
  namespace: production
spec:
  # Headless: no cluster IP — DNS returns individual pod IPs
  clusterIP: None
  selector:
    app: redis
  ports:
    - name: redis
      port: 6379
      targetPort: 6379

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
  namespace: production
spec:
  serviceName: redis-headless  # Must match the headless service name
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      # Init container: determine if this pod is the primary or replica
      initContainers:
        - name: configure-replication
          image: redis:7-alpine
          command:
            - sh
            - -c
            - |
              # Pod 0 is primary, others are replicas
              ORDINAL=$(hostname | awk -F'-' '{print $NF}')
              if [ "$ORDINAL" = "0" ]; then
                echo "This is the primary"
                cp /config/primary.conf /etc/redis/redis.conf
              else
                echo "This is a replica of redis-0"
                echo "replicaof redis-0.redis-headless.production.svc.cluster.local 6379" >> /etc/redis/redis.conf
              fi
          volumeMounts:
            - name: config
              mountPath: /etc/redis
            - name: redis-config
              mountPath: /config

      containers:
        - name: redis
          image: redis:7-alpine
          command: ["redis-server", "/etc/redis/redis.conf"]
          ports:
            - name: redis
              containerPort: 6379
          resources:
            requests:
              cpu: "100m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"
          volumeMounts:
            - name: data
              mountPath: /data
            - name: config
              mountPath: /etc/redis
          livenessProbe:
            exec:
              command: ["redis-cli", "ping"]
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            exec:
              command: ["redis-cli", "ping"]
            initialDelaySeconds: 5
            periodSeconds: 3

      volumes:
        - name: redis-config
          configMap:
            name: redis-config
        - name: config
          emptyDir: {}

  # VolumeClaimTemplates: each pod gets its OWN PVC
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]  # One pod per volume
        storageClassName: gp3-encrypted
        resources:
          requests:
            storage: 20Gi

StorageClass Configuration

# kubernetes/storage/gp3-storage-class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3-encrypted
  annotations:
    # Make this the default storage class
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer  # Don't provision until pod is scheduled
reclaimPolicy: Retain  # NEVER delete PVs automatically — use Delete only for dev
allowVolumeExpansion: true  # Allow resizing without deleting
parameters:
  type: gp3
  encrypted: "true"
  throughput: "125"   # MiB/s
  iops: "3000"

Stable Network Identity

Each StatefulSet pod gets a DNS entry:

Format: <pod-name>.<service-name>.<namespace>.svc.cluster.local

redis-0.redis-headless.production.svc.cluster.local → 10.0.1.5
redis-1.redis-headless.production.svc.cluster.local → 10.0.1.6
redis-2.redis-headless.production.svc.cluster.local → 10.0.1.7

// src/lib/redis-cluster.ts
// Connect to a specific Redis StatefulSet pod by ordinal

import { Redis } from "ioredis";

// Always connect to the primary (pod 0)
const primary = new Redis({
  host: "redis-0.redis-headless.production.svc.cluster.local",
  port: 6379,
});

// Read from replicas using round-robin
const REPLICA_COUNT = 2;
let replicaIndex = 0;

export function getReadReplica(): Redis {
  const ordinal = (replicaIndex % REPLICA_COUNT) + 1; // replicas are 1 and 2
  replicaIndex++;
  return new Redis({
    host: `redis-${ordinal}.redis-headless.production.svc.cluster.local`,
    port: 6379,
  });
}

PostgreSQL StatefulSet (Primary + Replica)

# kubernetes/statefulsets/postgres.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: production
spec:
  serviceName: postgres-headless
  replicas: 2  # postgres-0 = primary, postgres-1 = read replica
  podManagementPolicy: OrderedReady  # Default: wait for each pod to be Ready before starting next
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      partition: 1  # Update replicas first, then primary manually (safer)

  template:
    spec:
      containers:
        - name: postgres
          image: postgres:16-alpine
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: postgres-credentials
                  key: username
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-credentials
                  key: password
            - name: POSTGRES_DB
              value: production
            # Tell replicas where primary is
            - name: PRIMARY_HOST
              value: "postgres-0.postgres-headless.production.svc.cluster.local"
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
          lifecycle:
            preStop:
              exec:
                command:
                  - sh
                  - -c
                  - |
                    # Gracefully finish current transactions before terminating
                    ORDINAL=$(hostname | awk -F'-' '{print $NF}')
                    if [ "$ORDINAL" = "0" ]; then
                      psql -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE state = 'idle';"
                    fi

  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: gp3-encrypted
        resources:
          requests:
            storage: 100Gi

---
# Two services: one for primary writes, one for read replicas
apiVersion: v1
kind: Service
metadata:
  name: postgres-primary
  namespace: production
spec:
  selector:
    app: postgres
    # Route only to pod 0 using StatefulSet ordinal label
    statefulset.kubernetes.io/pod-name: postgres-0
  ports:
    - port: 5432

---
apiVersion: v1
kind: Service
metadata:
  name: postgres-replica
  namespace: production
spec:
  selector:
    app: postgres
    statefulset.kubernetes.io/pod-name: postgres-1
  ports:
    - port: 5432

PodDisruptionBudget for StatefulSets

# kubernetes/pdb/postgres-pdb.yaml
# Prevent both replicas from being disrupted simultaneously during upgrades
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: postgres-pdb
  namespace: production
spec:
  maxUnavailable: 1   # At most 1 pod can be unavailable at a time
  selector:
    matchLabels:
      app: postgres

Scaling and Lifecycle

# Scale a StatefulSet up — adds pods in order (2 → 3 → 4)
kubectl scale statefulset redis --replicas=4 -n production

# Scale down — removes pods in reverse order (4 → 3 → 2)
# Pods won't be deleted until the preceding pod is healthy
kubectl scale statefulset redis --replicas=2 -n production
# redis-3 deleted, then redis-2 deleted

# Rolling restart (e.g., after ConfigMap update)
kubectl rollout restart statefulset/redis -n production

# Check rollout status
kubectl rollout status statefulset/redis -n production

# Pause rollout if problems detected
kubectl rollout pause statefulset/redis -n production

# List PVCs (they persist after pod deletion)
kubectl get pvc -n production -l app=redis
# data-redis-0, data-redis-1, data-redis-2 — all persist

# Manually delete a PVC (only when you're sure you want to lose the data)
kubectl delete pvc data-redis-2 -n production

When NOT to Use StatefulSets on Kubernetes

Skip StatefulSets for:

1. Production databases (PostgreSQL, MySQL)
   Use managed services instead: RDS, Cloud SQL, PlanetScale
   Reason: Storage management, backups, failover are hard to get right yourself

2. Production message queues (Kafka, RabbitMQ)
   Use managed: MSK, Confluent Cloud
   Reason: Kafka tuning and rebalancing is complex

StatefulSets ARE appropriate for:
1. Redis clusters where you control the failure domain
2. Elasticsearch/OpenSearch (if not using managed)
3. Custom stateful applications with specific identity requirements
4. Development/staging environments of any database

See Also

• Kubernetes Cost Optimization — VPA + Karpenter for stateful workloads
• Kubernetes Networking — network policies for stateful pods
• Kubernetes RBAC — service accounts for database pods
• Database Replication — replication architecture

Working With Viprasol

Running stateful workloads on Kubernetes requires careful thought about storage classes, pod identity, disruption budgets, and when managed services are a better choice. Our platform engineers design stateful workload architectures that balance operational control with reliability, and know when to recommend RDS over a self-managed PostgreSQL StatefulSet.

Kubernetes engineering → | Talk to our engineers →