KubeDeploymentReplicasMismatch

warningKubernetes workloadsfree rule · MIT

Deployment has fewer available replicas than desired. Free rule (MIT): the complete runbook and rule definition are below.


Severity	warning
Pending (`for:`)	15m (warning)
Domain	Kubernetes workloads
Requires	kube-state-metrics v2.x
Rule file	`rules/kubernetes-workloads.rules.yml` (group `ftw.kubernetes-workloads.alerts`)

Meaning

Desired replicas are higher than available replicas and the rollout is not making progress (updated replicas unchanged for 10 minutes).

Impact

Reduced capacity and redundancy; an outage if available replicas reach zero.

Diagnosis

kubectl -n <namespace> get deployment <deployment> -o wide
kubectl -n <namespace> get pods -l <selector> -o wide

Unavailable pods usually show why: Pending (scheduling), CrashLoopBackOff, or failing readiness probes.

kube_deployment_spec_replicas{deployment="<deployment>"} - kube_deployment_status_replicas_available{deployment="<deployment>"}

Mitigation

Resolve the pod-level cause (see KubePodNotReady / KubePodCrashLooping).
If an HPA raised replicas beyond cluster capacity, add nodes or cap maxReplicas.
Roll back a release that broke readiness: kubectl -n <namespace> rollout undo deployment/<deployment>.

Escalation

warning: Notify the owning team's alert channel. Acknowledge within business hours; create a ticket if it cannot be resolved the same day. Escalate to critical handling if it trends toward the critical condition.

Route to the team that owns the namespace. Platform on-call only takes over when several namespaces are affected at once (likely a node, network or control-plane problem).

Related alerts

KubePodCrashLooping: Pod container is crash looping.
KubePodNotReady: Pod has been in a non-ready state.
KubeContainerOOMKilled: Container was OOM killed.
KubeImagePullBackOff: Container image cannot be pulled.
KubeDeploymentRolloutStuck: Deployment rollout is not progressing.

Rule definition

From rules/kubernetes-workloads.rules.yml in the free repository (MIT). Unit tests for it are in tests/.

- alert: KubeDeploymentReplicasMismatch
  expr: |-
    (kube_deployment_spec_replicas{job="kube-state-metrics", namespace=~".+"} > kube_deployment_status_replicas_available{job="kube-state-metrics", namespace=~".+"})
    and
    (changes(kube_deployment_status_replicas_updated{job="kube-state-metrics", namespace=~".+"}[10m]) == 0)
  for: 15m
  labels:
    severity: warning
  annotations:
    summary: Deployment has fewer available replicas than desired.
    description: Deployment {{ $labels.namespace }}/{{ $labels.deployment }} has had fewer available replicas than the desired {{ $value | humanize }} for longer than 15m, and no rollout is progressing.
    runbook_url: runbooks/kubernetes-workloads/KubeDeploymentReplicasMismatch.md

← all alert runbooks