// alert runbooks
165 Prometheus alerts, and what to check first
An alert fired and you have never seen it before. Every runbook here says what the alert actually means, what it breaks, the exact commands to diagnose it, and how to mitigate it — plus the rule definition that produced it. Free to read, no signup.
- Runbooks
- 165
- Alert domains
- 20
- Free MIT rules
- 12
- Guides
- 12
The rules behind these runbooks are tested
Every alert in the pack ships with a promtool unit test that proves it fires on a real failure, and a second that proves it stays quiet on healthy data. 12 of them are free and MIT-licensed — run the tests before you decide.
Hosts (node_exporter)
25- NodeClockNotSynchronising
- NodeClockSkewDetected
- NodeConntrackLimit
- NodeCPUHighIOWait
- NodeDiskIOSaturation
- NodeExporterDown
- NodeFileDescriptorLimit
- NodeFilesystemAlmostOutOfFiles
- NodeFilesystemAlmostOutOfSpace
- NodeFilesystemDeviceError
- NodeFilesystemSpaceFillingUp
- NodeHighCPUUsage
- NodeLoadHigh
- NodeMemoryHighUtilization
- NodeMemoryMajorPagesFaults
- NodeNetworkInterfaceFlapping
- NodeNetworkReceiveErrs
- NodeNetworkTransmitErrs
- NodeOOMKillDetected
- NodeRAIDDegraded
- NodeRAIDDiskFailure
- NodeRebootDetected
- NodeSystemdServiceFailed
- NodeTemperatureCritical
- NodeTextFileCollectorError
Prometheus self-monitoring
20- PrometheusConfigReloadFailed
- PrometheusDuplicateTimestamps
- PrometheusErrorSendingAlertsToAlertmanager
- PrometheusHighQueryLoad
- PrometheusLabelLimitHit
- PrometheusMissingRuleEvaluations
- PrometheusNotConnectedToAlertmanagers
- PrometheusNotificationQueueRunningFull
- PrometheusNotIngestingSamples
- PrometheusOutOfOrderTimestamps
- PrometheusRemoteStorageFailures
- PrometheusRemoteWriteBehind
- PrometheusRuleFailures
- PrometheusScrapeSampleLimitHit
- PrometheusSDRefreshFailure
- PrometheusTargetLimitHit
- PrometheusTSDBCompactionsFailing
- PrometheusTSDBReloadsFailing
- TargetDown
- Watchdog
Kubernetes workloads
13- KubeContainerOOMKilled
- KubeContainerWaiting
- KubeDaemonSetMisScheduled
- KubeDaemonSetNotScheduled
- KubeDaemonSetRolloutStuck
- KubeDeploymentGenerationMismatch
- KubeDeploymentReplicasMismatch
- KubeDeploymentRolloutStuck
- KubeImagePullBackOff
- KubePodCrashLooping
- KubePodNotReady
- KubeStatefulSetGenerationMismatch
- KubeStatefulSetReplicasMismatch
Kubernetes control plane & kubelet
12PostgreSQL (postgres_exporter)
10Redis (redis_exporter)
9etcd
8Kubernetes nodes & capacity
8Kubernetes quotas, limits & disruption budgets
7Alertmanager self-monitoring
6Apache Kafka (kafka_exporter)
6CoreDNS
6Endpoint probes (blackbox_exporter)
6Kubernetes certificates (kubelet & cert-manager)
6Grafana Loki
5Kubernetes persistent volumes
5NGINX Ingress Controller
5Kubernetes autoscaling (HPA)
4Kubernetes Jobs & CronJobs
3HTTP availability SLO (multi-window, multi-burn-rate)
1critical warning info in the free MIT edition