Environment variables in a Pod

Why env variables

The 12-factor app says something simple: anything that depends on the environment goes through environment variables. The database URL, the run mode, the key path, the API secret. The image stays the same, behavior changes through env.

In Kubernetes this is the job of the Pod field spec.containers[].env. From there, any process inside the container sees the variable through $DB_HOST in a shell, os.environ["DB_HOST"] in Python, os.Getenv("DB_HOST") in Go.

Minimal form

spec:
  containers:
    - name: app
      image: busybox:1.36.1
      env:
        - name: DB_HOST
          value: postgres-internal
        - name: LOG_LEVEL
          value: debug

env is an array of objects inside the container, at the same level as image. Each object requires name, and then either value (a string literal) or valueFrom (a reference to a ConfigMap, a Secret, or the downward API).

value is always a string. If you need a number or a boolean, quote the value:

- name: REPLICAS
  value: "3"
- name: DEBUG
  value: "true"

Without quotes, YAML parses 3 as a number, and the apiserver refuses to create the Pod:

Error from server (BadRequest): error when creating "STDIN": Pod in version "v1" cannot be handled as a Pod: json: cannot unmarshal number into Go struct field EnvVar.spec.containers.env.value of type string

valueFrom: four sources

value is just a string. valueFrom pulls the value from another cluster object. Four variants:

env:
  # from a ConfigMap
  - name: APP_MODE
    valueFrom:
      configMapKeyRef:
        name: app-config
        key: APP_MODE

  # from a Secret
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: app-secret
        key: password

  # from the downward API: the Pod's name, namespace, labels
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name

  # from the downward API: the container's requests/limits
  - name: CPU_LIMIT
    valueFrom:
      resourceFieldRef:
        containerName: app
        resource: limits.cpu

The downward API is a way to pass the container information about the Pod itself: its name, namespace, labels, limits. A Pod's name coming from a Deployment isn't known ahead of time (it has a random suffix), so it's taken through fieldRef instead of written as a string.

resourceFieldRef has an optional divisor field, defaulting to 1. The value is divided by it and rounded up to an integer. With limits.cpu: 100m, the variable CPU_LIMIT gets 1, while limits.memory: 32Mi gives 33554432 (bytes). If you need millicores, set divisor: 1m.

kubectl explain pod.spec.containers.env.valueFrom on v1.35 also lists a field fileKeyRef (a value from a file in a volume). It's a new feature, not used in this course.

The simplest form is a literal through value. ConfigMap and Secret as sources are covered in the next lessons of the track.

When env is read

Once, at container startup. You cannot change env on an existing Pod: in a running Pod, only a few fields can change, like image. kubectl edit or kubectl apply with a new env will be rejected by the apiserver:

The Pod "app" is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`,`spec.initContainers[*].image`,`spec.activeDeadlineSeconds`,`spec.tolerations` (only additions to existing tolerations),`spec.terminationGracePeriodSeconds` (allow it to be set to 1 if it was previously negative)

You need to delete the Pod and create it again (kubectl delete pod app, then kubectl apply -f), or do it in one command with kubectl replace --force -f app.yaml.

For a Deployment, recreation happens automatically: the template changes, the Deployment creates a new ReplicaSet, that brings up a Pod with fresh env, and the old ones get deleted.

For valueFrom from a ConfigMap, the same rule applies. If the ConfigMap changed, env in the running container stays old. You need a rollout. For dynamic updates, people use a volume mount: there, kubelet periodically refreshes the files.

Substituting variables in env

You can reference variables already declared earlier in the same array:

env:
  - name: HOST
    value: postgres-internal
  - name: PORT
    value: "5432"
  - name: DB_URL
    value: "postgres://$(HOST):$(PORT)/app"

The syntax is $(VAR), not ${VAR}: Kubernetes leaves ${HOST} alone. It only works for variables declared before the current one. If you mix up the order, $(HOST) stays a literal, and there's no error: the process just gets a string with parentheses in it. If you need the literal text $(HOST), escape it with double $$: $$(HOST) turns into $(HOST).

env, order of elements in the array:

  1  HOST   = postgres-internal
  2  PORT   = 5432
  3  DB_URL = $(HOST):$(PORT)   resolved
  4  BAD    = $(LATER)          literal
  5  LATER  = x

The same works in command and args:

command: ["sh", "-c"]
args: ["echo DB=$(DB_URL); sleep 3600"]

On top of that, the shell can expand $VAR on its own, so sh -c "echo $DB_HOST" also works. Here, Kubernetes isn't doing the substitution: the shell reads the env variable from its own environment.

Related commands

# Show the env of a specific container in the spec, one line per variable
kubectl get pod app -o jsonpath='{range .spec.containers[0].env[*]}{.name}={.value}{"\n"}{end}'

# Run env inside the Pod and see the actual values
kubectl exec app -- env | sort

# See what the container printed on startup
kubectl logs app

# Look up the field schema
kubectl explain pod.spec.containers.env
kubectl explain pod.spec.containers.env.valueFrom --recursive

For variables from valueFrom, the value field is empty, so the jsonpath above will show POD_NAME=. The actual value is only visible through kubectl exec ... env.

What often breaks

• A number without quotes: value: 3 → a parse error on EnvVar.Value. Quote the values.
• Env changed, the Pod didn't pick it up. Env isn't reactive, you need to recreate the Pod.
• A reference to $(VAR), but VAR is declared lower in the list. The substitution won't fire, the process gets the literal $(VAR).
• env written at the Pod's spec level instead of the container's: kubectl apply replies unknown field "spec.env".

What you'll need for the tasks

• Lab 1 (labs/01-shop-env): an env block with value, quotes for the number, and valueFrom.fieldRef from the section "valueFrom: four sources".
• Lab 2 (labs/02-orders-db-url): the substitution order of $(VAR) and how to change env on an already created Pod.
• Commands: kubectl apply -f, kubectl logs, kubectl exec ... env, kubectl delete pod.

Homework

Practice the theory on your own cluster: 3 steps with a check. Set up the cluster with the Kubernetes locally track, the tasks live in the k8s-practice repository.

./run.sh start k8s-env-vars     # set up the starting state
./run.sh verify k8s-env-vars    # check the current step

In short

• Env is set in spec.containers[].env, each element is a name plus value or valueFrom.
• value is always a string, numbers and booleans are quoted.
• valueFrom pulls a value from a ConfigMap, a Secret or the downward API.
• Env is read once at container startup, an update requires recreating the Pod.
• The $(VAR) substitution only works for variables declared earlier in the list.