Volumes and environment variables

Why

From the lesson on images, you know that a container is a writable layer on top of a read-only stack of image layers. Everything a process writes to that layer without a volume gets deleted along with the container on docker rm.

For a web server or a stateless service, that's fine. For a database that accumulates data over weeks, that's data loss on the very first container recreation.

The solution is a mount: external storage attached to a path inside the container. There are three kinds of mounts. The second half of the lesson covers environment variables and the mistake almost everyone trips over: localhost inside a container.

Three kinds of mounts

Type	Where data is written	Survives docker rm	Usually for
named volume	Docker's own directory (/var/lib/docker/volumes/...)	yes	databases, any state
bind mount	a host path you specify	yes	development: source code, configs
tmpfs	RAM, not disk	no, disappears already on docker stop	secrets, temporary cache

On macOS and Windows, the /var/lib/docker/volumes directory is inside Docker Desktop's Linux VM, not in your own file system. Refer to a volume by name, not by path.

Named volume

docker volume create pgdata
docker run -d --name db -e POSTGRES_PASSWORD=example \
  -v pgdata:/var/lib/postgresql/data postgres:16.4-alpine

Docker manages the directory on the host itself, you refer to the volume by name. The volume survives docker rm db. To attach the same data to a new container, specify the same -v pgdata:/var/lib/postgresql/data.

Use a named volume by default for anything that stores state: postgres, redis with disk persistence, any service that writes data.

Bind mount

docker run -d --name dev -v "$PWD/src":/app/src node:22.11.0-alpine sleep infinity

To the left of the colon is a host path (absolute, hence $PWD), to the right the path inside the container. This is handy for development: you edit files in your IDE on the host, and the container sees the changes right away.

The subtle part here is permissions. The process inside the container runs under its own UID, and that UID needs permission on the host files. In the postgres:16.4-alpine image, user postgres has uid 70 (in the debian variant postgres:16.4 it's 999). If you hand it a bind mount to a directory owned by your own uid 1000, postgres won't be able to write there and will fail with permission denied. A fresh, empty named volume doesn't have this problem: on first mount, Docker copies that path's contents from the image into the volume, along with the owner.

tmpfs

docker run --rm -it --tmpfs /tmp:size=100m,mode=1777 alpine:3.20.3 sh

Files sit in RAM and are never written to disk. When the container stops, they disappear, even if you later start it again with docker start. tmpfs fits two cases:

• a temporary cache you don't mind losing;
• secrets the app reads at startup that must never land on disk.

VOLUME in the Dockerfile and anonymous volumes

If the image's Dockerfile has VOLUME /var/lib/postgresql/data (the official postgres image does), then docker run creates an anonymous volume for that path on its own, even if you didn't specify anything.

It works the same way as a named volume, except instead of a name it has a random 64-character hex string. You can only attach that volume to a new container using that string. Usually nobody does, and anonymous volumes pile up.

This is where the classic data loss comes from. You started a database without -v, then recreated the container with the same command. The new container got a new, empty anonymous volume. The old data didn't go anywhere, it's sitting in the old volume, but the database can't see it.

See the volumes:

docker volume ls

DRIVER    VOLUME NAME
local     6295a854674df457df2c0263592e37646cd200a7f2af0082a71aab7944610914
local     pgdata

Which volume is attached to a specific container is shown by the Mounts section:

docker inspect -f '{{range .Mounts}}{{.Type}} {{.Name}} -> {{.Destination}}{{println}}{{end}}' db

Clean up like this:

docker container prune    # remove stopped containers first
docker volume prune       # then volumes; one attached to a container is left alone

docker rm -v <container> removes a container together with its anonymous volumes. Named volumes stay in place.

There's a nuance here that trips up even people who've worked with Docker for a long time. Starting with Docker Engine 23.0, docker volume prune by default removes only anonymous volumes. It leaves unused named ones alone. To take those too:

docker volume prune -a     # anonymous + unused named

Careful: -a will also catch a forgotten pgdata from a project you haven't started in a month. Check docker volume ls first. On Docker 20.10 and 22.x, there's no -a flag, and plain docker volume prune removes all unused volumes, named ones included.

There's also a catch-all command:

docker system prune --volumes    # containers, networks, untagged images, anonymous volumes

Build cache is counted separately from images and volumes and can take up tens of gigabytes:

docker system df          # Images, Containers, Local Volumes, Build Cache rows
docker builder prune      # cleans the build cache

Backing up a volume with a disposable container

You can pull data out of any volume, anonymous ones included, through a temporary container. Substitute your own volume's name from docker volume ls for 6295a854...:

docker run --rm \
  -v 6295a854674df457df2c0263592e37646cd200a7f2af0082a71aab7944610914:/data:ro \
  -v "$PWD":/backup \
  alpine:3.20.3 tar czf /backup/dump.tgz /data

The volume is mounted read-only (:ro), the current host folder for writing. tar packs the data, the container is removed (--rm), and dump.tgz is left on the host. Restoring works the same way, just with tar xzf and the volume without :ro.

The same trick moves data from one volume to another: attach both and copy with cp -a, the -a flag preserves file ownership and permissions.

Copying files from a running database is risky: the copy can end up inconsistent. Stop the database container before copying, or dump with the database's own tools (pg_dump). This is covered in detail in the backups track.

Environment variables

You can set a container's variables in three ways.

The -e KEY=VAL flag, one at a time:

docker run -e DB_HOST=db -e DB_PORT=5432 myapp:1.0

The --env-file flag reads KEY=VAL pairs from a file:

docker run --env-file app.env myapp:1.0

In the Dockerfile, the ENV instruction. This is a default value, and -e and --env-file override it. The ENV value is baked into the image: it's visible in docker history and docker image inspect. So secrets don't go through ENV.

In compose, the same thing is done by a service's environment: and env_file: blocks. After editing variables, docker compose up -d recreates the container with the new values. docker compose restart doesn't do this: it restarts the old container with the old environment.

The classic trap: localhost inside a container

This is the most common mistake. In the app code:

const dbHost = process.env.DB_HOST || 'localhost';
const dbPort = process.env.DB_PORT || 5432;

In compose:

services:
  api:
    image: myapp:1.0
    # environment not set
  db:
    image: postgres:16.4-alpine

You start it and get ECONNREFUSED on localhost:5432. Why?

DB_HOST isn't set, and the client took the default value localhost. And localhost inside a container means this very container, not the host and not a neighboring container. Every container has its own network namespace with its own loopback interface. Nobody is listening on port 5432 inside api.

The fix in compose:

services:
  api:
    image: myapp:1.0
    environment:
      DB_HOST: db                 # service name
      DB_PORT: 5432
    depends_on: [db]
  db:
    image: postgres:16.4-alpine
    environment:
      POSTGRES_PASSWORD: example

Here db is the service name in compose. Compose makes it a DNS name in the project's shared network (covered in detail in the lesson on networking), so api connects to db:5432 and lands in the database container.

Don't edit the code to remove the fallback to localhost. Running without compose, on your own machine, is still a normal scenario, and the default value is needed there. Fix the configuration instead.

Secrets

What not to do:

• Don't put a .env with real keys into Git. A .env* line in .gitignore is mandatory.
• Don't pass secrets through ENV or ARG in a Dockerfile: they stay in the image's history and config.
• Don't put secrets in image tags or container names.

What to do in production:

• Docker secrets (Swarm, as well as secrets: in compose: the secret is mounted as a file at /run/secrets/<name>) or a Kubernetes Secret.
• An external secrets store: HashiCorp Vault, AWS Secrets Manager, Google Secret Manager. Secrets live there, the app gets them by token.
• Encrypting secrets right in the repository: sops encrypts the values in the file, the key is kept separately.
• The minimal option: --env-file from a file that isn't tracked in Git and lives on the server with 600 permissions.

Common mistakes

Recreated the database container, data is gone. The database was started without -v, and the data lived in an anonymous volume. Find the old volume with docker inspect on the old container (the Mounts section) or docker volume ls. Move the data into a named volume and run the database only with that from now on.

Typo in a bind mount path. With the -v flag, Docker silently creates an empty directory on the host at a path that doesn't exist, and the container sees an empty folder instead of your files. The --mount type=bind,src=...,dst=... flag fails with an error in this situation, so the typo shows up right away.

DB_HOST=localhost in a container. Covered in the section above. You need the service name to reach a neighboring service.

depends_on doesn't wait for readiness. Without conditions, depends_on only sets the start order. Container api starts once container db is created and running, but postgres inside it may still be initializing. The app should reconnect, or you need depends_on with condition: service_healthy and a healthcheck on the database.

A .env file next to compose doesn't make it into the container. Compose reads it to substitute ${VAR} in docker-compose.yml itself. To get variables into the container, list them under environment: or wire up the file with env_file:.

Changed env_file and ran docker compose restart. The container restarted with the old environment. You need docker compose up -d.

What you'll need for the tasks

• docker volume create, docker volume ls, docker volume rm
• docker run -v <volume>:<path in container> and :ro
• docker inspect <container>, the Mounts section
• docker rm and docker rm -v
• a disposable container, docker run --rm with two volumes and cp -a
• docker exec <container> env, to see the variables inside
• env_file: and environment: in compose, docker compose up -d, docker compose logs
• curl -sS http://127.0.0.1:PORT/

The next lesson covers Docker networking and why a service name works as a DNS name.