Kustomize: Secret generator

Why

A database password, an API token, a certificate's private key: none of these belong in a ConfigMap next to a button color. Kubernetes has a separate object for them, and Kustomize builds it the same way it built the ConfigMap in the last lesson.

Secret is the Kubernetes object for sensitive data: passwords, tokens, keys, and certificates. It works almost like ConfigMap. It differs in object type, in having separate access rules, and in storing values in the data field encoded as base64.

secretGenerator follows the same rules as configMapGenerator: the same three sources (literals, files, envs), the same hash suffix, the same generatorOptions block.

Generating from literals

secretGenerator:
  - name: app-secret
    literals:
      - DB_PASSWORD=s3cr3t
      - API_TOKEN=abc123

Write values as plain text. You don't encode them in base64 by hand: Kustomize does that at render time. Encode them ahead of time and Kustomize will encode the already-encoded string again. The result is a Secret of type Opaque:

apiVersion: v1
data:
  API_TOKEN: YWJjMTIz
  DB_PASSWORD: czNjcjN0
kind: Secret
metadata:
  name: app-secret-ddg88mbgh9
type: Opaque

Secret from files

For certificates and keys, files is more convenient: a PEM file is multi-line, and you can't paste it into literals.

secretGenerator:
  - name: web-tls
    type: kubernetes.io/tls
    files:
      - tls.crt=certs/site.pem
      - tls.key=certs/site.key

The type field sets the Secret's type. Without it you get a plain Opaque. The kubernetes.io/tls type requires the keys tls.crt and tls.key, and by default the key becomes the file name. That's why files get renamed on the way in: to the left of = is the key name, to the right is the file path.

Kustomize doesn't check type requirements. A Secret with keys site.pem and site.key will build without a single error, and only the API server will reject it, at kubectl apply.

There's also envs: it parses a KEY=value file line by line, and each line becomes a separate key. Useful when a set of variables already lives in an .env file:

secretGenerator:
  - name: app-env
    envs:
      - secret.env

files with that same secret.env would give you one key, secret.env, holding the entire file.

base64 is encoding, not encryption

Values in a Secret look like random text, and it's easy to mistake that for protection. There isn't any. base64 is a reversible encoding, built for passing binary data as text. Anyone with permission to read the Secret gets the original value back with one command:

kubectl get secret app-secret-ddg88mbgh9 -o jsonpath='{.data.DB_PASSWORD}' | base64 --decode

s3cr3t

base64 --decode works the same on macOS and Linux. The same trick works without a cluster too, on the output of kubectl kustomize.

What actually limits access:

• RBAC decides who can even run get secret in that namespace.
• etcd encryption at rest locks down the data on the control plane nodes' disks.
• External stores (Vault, cloud secret managers) keep values outside the cluster, and only a reference or a short-lived token ever reaches Kubernetes.

The practical value of Secret is that it separates sensitive data from the rest of the configuration and gives it its own access-control point. The object itself doesn't encrypt the content.

The hash suffix

Just like ConfigMap, a content hash gets appended to the name: app-secret-ddg88mbgh9. Change the value, and the name changes too. Kustomize fixes up the reference in the Deployment, the Pod template becomes different, and the cluster starts a rollout: Pods get recreated and pick up the new password.

Turn it off the same way:

generatorOptions:
  disableNameSuffixHash: true

Without the hash, the Secret's name is fixed. After you change the password, kubectl apply -k updates the data in the Secret, but the Pod template in the Deployment doesn't change, no rollout starts, and the running Pods keep the old environment variables until you restart them by hand.

What usually breaks

A password in literals lands in git in plain text. That's the main problem with secretGenerator, and Kustomize doesn't solve it. Repositories keep either encrypted files (SOPS, sealed-secrets) or just a reference to an external store, and leave literals for local experiments.

Don't render for debugging on a step whose logs are publicly readable.

A manual manifest and a generator with the same name. If resources still has secret.yaml with a Secret app-secret, and there's a generator named app-secret right next to it, the build fails:

error: merging from generator &{...}: id resid.ResId{Gvk:resid.Gvk{Group:"", Version:"v1", Kind:"Secret", isClusterScoped:false}, Name:"app-secret", Namespace:""} exists; can not use behavior: 'unspecified', behavior must be merge or replace

Moving a Secret to the generator? Remove the manual file from resources.

Changing a Secret's type on a live object doesn't work: the type field is immutable. If a Secret was created as Opaque and you later decide to make it kubernetes.io/tls, you'll have to delete the object and create it again. With the hash suffix enabled, this usually isn't a problem: the new type changes the hash, and an object with a different name gets created.

What you'll need for the tasks

• Lab 01 (Build): replace a manual base64 Secret with a generator reading an .env file, and build a TLS Secret from a certificate and a key.
• Lab 02 (Fix): find why the Secret's password doesn't match, why the API server rejects the TLS Secret, and why changing the password doesn't restart the Pod.
• Commands: kubectl kustomize <folder>, grep over the output, base64 --decode, od -c (shows invisible characters), openssl version. No cluster or Docker needed.

Homework

Practice the theory on your own cluster: 2 steps with a check. Set up the cluster with the Kubernetes locally track, the tasks live in the k8s-practice repository.

./run.sh start k8s-kustomize-secret-generator     # set up the starting state
./run.sh verify k8s-kustomize-secret-generator    # check the current step

In short

• secretGenerator works like configMapGenerator: the same literals, files, envs, and hash suffix.
• Values are written in plain text; Kustomize adds the base64 itself.
• The type field sets the Secret's type, for example kubernetes.io/tls with the keys tls.crt and tls.key. Kustomize doesn't check types; the API server does.
• files places the file as is, trailing newline included.
• base64 decodes with one command; RBAC, etcd encryption, and external stores are what actually protect it.
• Passwords in literals sit in git in plain text, which is why real repositories use SOPS, sealed-secrets, or an external secret manager.