Kubernetes Probes: Liveness, Readiness, and Startup

Kubernetes Probes: Liveness, Readiness, and Startup

Kubernetes liveness, readiness, and startup probes are like health check mechanisms for your applications running in containers, ensuring theyโ€™re operating smoothly. Hereโ€™s a simple explanation of each using an analogy of running a business.

๐—Ÿ๐—ถ๐˜ƒ๐—ฒ๐—ป๐—ฒ๐˜€๐˜€ ๐—ฃ๐—ฟ๐—ผ๐—ฏ๐—ฒ (๐—œ๐˜€ ๐˜๐—ต๐—ฒ ๐˜€๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ฐ๐—ฒ ๐˜€๐˜๐—ถ๐—น๐—น ๐—ฎ๐—น๐—ถ๐˜ƒ๐—ฒ?)

๐—”๐—ป๐—ฎ๐—น๐—ผ๐—ด๐˜†: Imagine youโ€™re running a coffee shop. The liveness probe is like checking whether your coffee machine is still functioning throughout the day. If the machine breaks down and stops making coffee, you need to either fix it or replace it, otherwise, customers wonโ€™t get coffee.

๐—œ๐—ป ๐—ž๐˜‚๐—ฏ๐—ฒ๐—ฟ๐—ป๐—ฒ๐˜๐—ฒ๐˜€: The liveness probe ensures that your application is still alive and not stuck or crashed. If it fails, Kubernetes restarts the container automatically. For example, if a container goes into a deadlock, the liveness probe detects this and restarts the container to fix the issue.

๐—ช๐—ต๐—ฒ๐—ป ๐˜๐—ผ ๐˜‚๐˜€๐—ฒ: If your application can sometimes get into an unrecoverable state, like a crash or deadlock, this probe can automatically fix it by restarting the container.

๐—ฅ๐—ฒ๐—ฎ๐—ฑ๐—ถ๐—ป๐—ฒ๐˜€๐˜€ ๐—ฃ๐—ฟ๐—ผ๐—ฏ๐—ฒ (๐—œ๐˜€ ๐˜๐—ต๐—ฒ ๐˜€๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ฐ๐—ฒ ๐—ฟ๐—ฒ๐—ฎ๐—ฑ๐˜† ๐˜๐—ผ ๐˜€๐—ฒ๐—ฟ๐˜ƒ๐—ฒ ๐—ฐ๐˜‚๐˜€๐˜๐—ผ๐—บ๐—ฒ๐—ฟ๐˜€?)

๐—”๐—ป๐—ฎ๐—น๐—ผ๐—ด๐˜†: In the coffee shop, the readiness probe checks if your coffee machine is ready to serve customers. For example, after turning on the machine, it might take a few minutes to warm up before it can make coffee. If the machine isnโ€™t ready yet, customers shouldnโ€™t be directed to it.

๐—œ๐—ป ๐—ž๐˜‚๐—ฏ๐—ฒ๐—ฟ๐—ป๐—ฒ๐˜๐—ฒ๐˜€: The readiness probe checks if the container is ready to handle traffic. It might take time for your app to fully initialize or connect to dependencies (like a database). Kubernetes wonโ€™t send any traffic to your app until the readiness probe passes.

๐—ช๐—ต๐—ฒ๐—ป ๐˜๐—ผ ๐˜‚๐˜€๐—ฒ: Use it when your application needs some time to initialize before being able to serve requests (e.g., loading configuration, warming up cache).

๐—ฆ๐˜๐—ฎ๐—ฟ๐˜๐˜‚๐—ฝ ๐—ฃ๐—ฟ๐—ผ๐—ฏ๐—ฒ (๐—œ๐˜€ ๐˜๐—ต๐—ฒ ๐˜€๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ฐ๐—ฒ ๐˜‚๐—ฝ ๐—ฎ๐—ป๐—ฑ ๐—ฟ๐˜‚๐—ป๐—ป๐—ถ๐—ป๐—ด ๐—ฎ๐˜ ๐˜€๐˜๐—ฎ๐—ฟ๐˜๐˜‚๐—ฝ?)

๐—”๐—ป๐—ฎ๐—น๐—ผ๐—ด๐˜†: When you first open the coffee shop in the morning, the startup probe is like checking if everything (coffee machine, lights, etc.) is set up and ready for business. Until this initial setup is complete, no customers can come in.

๐—œ๐—ป ๐—ž๐˜‚๐—ฏ๐—ฒ๐—ฟ๐—ป๐—ฒ๐˜๐—ฒs: The startup probe checks if your containerโ€™s application has started successfully. Itโ€™s especially useful for apps that have slow startup times. Until the startup probe passes, Kubernetes wonโ€™t send any other probes (liveness or readiness), ensuring that the app doesnโ€™t get prematurely killed or sent traffic before itโ€™s ready.

๐—ช๐—ต๐—ฒ๐—ป ๐˜๐—ผ ๐˜‚๐˜€๐—ฒ: Ideal for apps that have a long initialization process. This prevents Kubernetes from misjudging slow-starting apps as unhealthy.

Example in Node.js API

Here's how to set up liveness, readiness, and startup probes for a Node.js API in Kubernetes

Suppose you have a simple Node.js API with an endpoint /health for health checks.

// server.js
const express = require("express");
const app = express();
const PORT = process.env.PORT || 3000;

let isReady = false; // Used to simulate readiness

app.get("/health", (req, res) => {
  res.status(200).send("API is alive");
});

app.get("/readiness", (req, res) => {
  if (isReady) {
    res.status(200).send("API is ready");
  } else {
    res.status(503).send("API is not ready");
  }
});

// Simulate delayed readiness
setTimeout(() => {
  isReady = true;
}, 10000); // API becomes ready after 10 seconds

app.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});

This API has

  • /health: Returns 200 OK to indicate the API is alive.

  • /readiness: Returns 503 Service Unavailable until the API is fully ready, then returns 200 OK after 10 seconds.

Kubernetes Probes

  1. Liveness Probe: Checks if the app is alive by hitting /health. If this probe fails, Kubernetes restarts the container.

  2. Readiness Probe: Checks if the app is ready to handle traffic by hitting /readiness. Only after the app is fully initialized does this probe return a 200 OK, allowing Kubernetes to route traffic to it.

  3. Startup Probe: For apps with a slow start, the startup probe waits until the app is completely initialized before allowing other probes to execute.

Hereโ€™s how you can define these probes in your Kubernetes deployment YAML file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodejs-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodejs-api
  template:
    metadata:
      labels:
        app: nodejs-api
    spec:
      containers:
      - name: nodejs-api
        image: node:14
        command: ["node", "server.js"]
        ports:
        - containerPort: 3000
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /readiness
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 5
        startupProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 5
          failureThreshold: 20

Explanation of Each Probe Configuration

  • Liveness Probe:

    • Path: /health

    • initialDelaySeconds: Waits 5 seconds before the first check.

    • periodSeconds: Checks every 10 seconds to see if the app is alive. If it fails, Kubernetes restarts the container.

  • Readiness Probe:

    • Path: /readiness

    • initialDelaySeconds: Waits 5 seconds before the first check.

    • periodSeconds: Checks every 5 seconds to see if the app is ready to serve traffic. Traffic isnโ€™t sent until this probe passes.

  • Startup Probe:

    • Path: /health

    • initialDelaySeconds: Waits 5 seconds before the first check.

    • periodSeconds: Checks every 5 seconds during startup.

    • failureThreshold: Allows up to 20 checks (100 seconds) to ensure the app has fully initialized before liveness or readiness probes start, preventing premature restarts.

This setup helps Kubernetes manage your Node.js APIโ€™s health effectively, restarting it if it crashes, waiting until itโ€™s ready before routing traffic, and allowing time for complete startup if initialization is slow.

Which probes failled restart the container?

  1. Liveness Probe: If the liveness probe fails, Kubernetes restarts the container. This probe checks if the application is still "alive" (not in a crashed or deadlocked state). If it detects an issue, it considers the container unhealthy and restarts it to try to resolve the issue.

  2. Readiness Probe: A readiness probe failure does not restart the container. Instead, it marks the container as "unready," meaning Kubernetes will stop routing traffic to it until the probe passes again. This is useful if the application temporarily cannot serve requests (e.g., during maintenance or reconfiguration).

  3. Startup Probe: If the startup probe fails, it also triggers a container restart. The startup probe is intended for applications that need extra time to initialize. It prevents the liveness and readiness probes from prematurely failing until the app has fully started up.

So, only the liveness and startup probes trigger a restart (Also depending on the Restart Policy default to Always ) when they fail. The readiness probe only affects traffic routing.

Execution of of a probes

In Kubernetes, probes are checked in a specific sequence based on the presence of a startup probe

  1. Startup Probe (if configured): This probe is checked first. When a startup probe is defined, Kubernetes will wait until it passes successfully before starting to run the liveness and readiness probes. This avoids premature restarts or traffic routing failures for applications with slow initialization. While the startup probe is running, the liveness and readiness probes are disabled.

  2. Liveness and Readiness Probes (after Startup Probe passes):

    • Liveness Probe: Checks if the container is still healthy and able to function.

    • Readiness Probe: Determines if the container is ready to serve traffic.

Once the startup probe passes (or if it isnโ€™t defined), the liveness and readiness probes begin their checks according to their configured intervals.

Priority Summary

  • Startup Probe has the highest priority. Until it succeeds, the liveness and readiness probes are ignored.

  • Liveness and Readiness Probes then run independently of each other on their own schedules once the startup probe has passed.

Summary

Liveness Probe restarts the container on failure.

Readiness Probe controls traffic routing based on readiness.

Startup Probe provides a grace period for slow-starting apps before liveness and readiness checks.

Sources

https://www.gremlin.com/blog/how-to-ensure-your-kubernetes-pods-and-containers-can-restart-automatically

https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Did you find this article valuable?

Support Beyond DevOps by becoming a sponsor. Any amount is appreciated!

ย