Why Kubernetes

You have 4 api containers on two servers. At 4am one server goes down. With Docker and no orchestrator, you get an alert in the middle of the night and run to SSH into the servers. With Kubernetes (k8s), you find out about the crash in the morning from the logs, because the cluster already started a replacement on its own.

Below are two situations k8s was built for. First: a node dies. Second: a big traffic spike hits because of an ad campaign. Then we'll cover when you don't need k8s.

Concepts

Five words are enough for now. We'll cover each one in detail in its own lesson.

• Node — a single machine in the cluster that runs containers.
• Pod — the smallest unit Kubernetes runs: one or more containers sharing an IP.
• Deployment — an object where you write "I want 4 copies of a Pod from this image." The copies are called replicas.
• Controller — a process in the cluster that compares the desired state with the real one and fixes the difference. A Deployment has two: the Deployment controller creates a ReplicaSet, and the ReplicaSet controller keeps the right number of Pods.
• Scheduler — the component that picks which node to put a new Pod on.

Scene 1: a node goes down

Kubelet, the k8s agent on every node, reports to the cluster regularly. If there are no reports for 50 seconds, the node controller sets the node's Ready condition to Unknown. This is the default value of the --node-monitor-grace-period flag on kube-controller-manager, and you can change it on your own cluster.

Within that same minute, Pods on that node stop counting as ready, and the Service removes them from its list of addresses. Traffic only goes to live Pods. Requests that already went to the dead node before this point get an error, so the client needs retries.

k8s doesn't start a replacement right away. The node gets a taint (a "don't schedule here" mark), and by default every Pod has a toleration that lets it wait out that mark for 300 seconds, in case the node comes back. After 5 minutes, Pods on the dead node are deleted. The ReplicaSet sees "should be 4, there are 2" and creates two new Pods. The scheduler places them on a node with enough CPU and memory for the Pod's resource requests (requests). Kubelet on that node pulls the image if it's not there yet and starts the containers.

In total, about 6 minutes pass from the crash to full recovery, and nobody does anything by hand. You deal with the dead node in the morning, with a clear head.

Scene 2: traffic arrives

This is handled by HPA (Horizontal Pod Autoscaler). Every 15 seconds it looks at Pod metrics and recalculates how many replicas are needed to bring the average load back to the target. If more are needed, HPA changes the replica count in the Deployment, and the same ReplicaSet and scheduler from the first scene take it from there.

HPA gets CPU and memory metrics from the Metrics Server add-on. It's installed into the cluster separately: managed clusters usually already have it, the lab's kind cluster doesn't. Other metrics, for example RPS from Prometheus, are connected through the custom and external metrics APIs.

The peak is over and load drops below the target. HPA doesn't remove Pods right away: by default it watches for 5 minutes to see if load comes back, and only then reduces the replica count. That way the cluster doesn't jerk around on short spikes.

Declare state, not commands

Both scenes work on the same principle. You don't tell the cluster "start two more containers." You write how many replicas you want into the manifest and apply it:

kubectl apply -f api.yaml

From there, the controller keeps reality matching what's written. A Pod crashes, you delete a Pod by hand, a node disappears: the controller sees the difference and creates a replacement. With Docker and no orchestrator, you do the same thing yourself: you log into the server and run docker run again.

The manifest in git becomes the single source of truth. If you change the replica count with the kubectl scale command, the cluster obeys, but the file still has the old number. The next kubectl apply brings back the old value, and you'll wonder where your replicas went.

When you do NOT need k8s

Most projects don't need k8s. It pays off for companies with dozens of services, or where traffic spikes sharply (the same ad campaign from the second scene). Below that threshold, it costs more than it saves.

One container on one VPS: docker run and systemd. Several services with a database: docker compose. For a landing page, nginx is enough. On tasks like these, k8s takes more time than it gives back.

The cluster itself needs maintenance too: control plane, ingress, RBAC, etcd backups, monitoring. If you can't explain to yourself in one sentence why compose isn't enough, compose is probably enough for you. If you do go with k8s, a managed cluster from a cloud provider (for example, AWS, GCP or DigitalOcean) is usually cheaper and simpler than running the control plane yourself.

Common mistakes

• Expecting recovery within a minute. The cluster notices the node in 50 seconds, but only recreates its Pods after 5 minutes. If a service needs to survive a node failure without a dip, spread replicas across nodes ahead of time.
• Setting up HPA without Metrics Server. HPA gets created with no errors, but the replica count never changes. kubectl get hpa shows cpu: <unknown>/50% in the TARGETS column, and kubectl describe hpa prints FailedGetResourceMetric.
• Fixing by hand what's described in the manifest. kubectl scale and kubectl edit work, but the next kubectl apply from git brings everything back to how it was.
• Thinking k8s moves Pods around. Pods don't move. Old ones get deleted, new ones get created with new names and IPs. That's why clients go through a Service instead of a Pod's IP.

What you'll need for the tasks

• Lab 1 brings up a learning kind cluster on your laptop. You need Docker, kind and kubectl. As for commands, kubectl get pods, kubectl delete pod <name> and kubectl apply -f <file> are all you need.
• Lab 2 needs no cluster: just a browser and Kubernetes documentation pages.

In the next lesson, we'll cover how the cluster is put together: the control plane (api-server, etcd, scheduler, controller-manager) and worker nodes with kubelet.