The container crashed: how to figure out why

Why

A service didn't come up, and docker ps is empty. Or it's stuck in restarts, or it's running but not responding. Guessing and rebuilding the image at random takes forever. Docker has everything you need to find the cause in a couple of minutes: status and exit code, logs, state in docker inspect, a stream of events, and ways to look inside an image even when it has no shell.

This lesson goes through these tools in order, and the typical mistakes they point to.

Concepts

Exit code — a number from 0 to 255 the container's main process exits with. Zero means success, anything else means an error.

Main process — the command from the image's ENTRYPOINT and CMD (or from docker run). The container lives as long as this process does.

Shebang — the first line of a script, in the form #!/bin/sh. It tells the kernel which program to run the script with.

Namespace — a kernel mechanism that gives a container its own process list and its own network. You can attach another container to a running container's namespace.

The container disappeared from the list

A typical picture: you start a service, check docker ps, and it isn't there. Try it yourself, it's safe:

docker run -d --name api alpine:3.20.3 sh -c 'echo "Error: connect ECONNREFUSED 127.0.0.1:5432" >&2; exit 1'
docker ps

CONTAINER ID   IMAGE   COMMAND   CREATED   STATUS   PORTS   NAMES

The list is empty, but the container didn't go anywhere. docker ps shows only running containers, and this one started and immediately exited. Show all of them:

docker ps -a

CONTAINER ID   IMAGE           STATUS                     NAMES
3f2a1c9e8b7d   alpine:3.20.3   Exited (1) 5 seconds ago   api

Exited (1) is already a clue: the number in parentheses is the main process's exit code. Start there.

What the exit code tells you

A process on Linux, when it finishes, returns a number from 0 to 255. Zero means success, any other number means an error.

Codes above 128 are special. By Unix convention, if a process didn't exit on its own but was killed by a signal, the code equals 128 plus the signal number. kill -9 sends signal 9, so a process killed that way returns 137.

Code	What it usually means
0	the process ran to completion and exited on its own
1	an error in the application, check docker logs
125	docker run itself failed: a typo in a flag, a name already in use
126	the file from CMD exists but won't run: missing x permission
127	the command from CMD isn't in the image
137	killed by signal 9 (SIGKILL)
143	killed by signal 15 (SIGTERM), usually via docker stop

Codes 125, 126, and 127 have a catch. docker run itself returns them, because the process inside never actually started. With 126 and 127 the container is created but stays in status Created rather than Exited, and its docker logs is empty. With 125 the container usually isn't created at all. For example, alpine has no bash:

docker run --name nobash alpine:3.20.3 bash

docker: Error response from daemon: ... exec: "bash": executable file not found in $PATH: unknown.

echo $?

127

In a case like this, docker inspect shows the cause, more on that below.

Code 137 happens for three reasons: the container hit a memory limit, it was killed with docker kill, or docker stop waited 10 seconds for it to finish and then killed it anyway. docker inspect will show which of the three it was. Why stop has to finish the job itself is something we'll cover in the next lesson.

A word about 0. A container lives as long as its main process lives. Run docker run -d alpine:3.20.3, and the container exits with code 0 within a fraction of a second. The alpine image's main command is sh, and without input attached (the -i flag) it has nothing to read, so it exits. That's not an error. If your service behaves like this, check which command is baked into the image:

docker inspect alpine:3.20.3 --format '{{.Config.Cmd}}'

[/bin/sh]

You'll get the same code 0 if the main process starts a server in the background and then finishes on its own. Many servers daemonize by default, and then the container's main process ends along with the startup script. A server inside a container needs to run in the foreground.

Logs of a crashed container

docker logs works on a stopped container too. Everything the process managed to write to stdout and stderr is still there:

docker logs api

Error: connect ECONNREFUSED 127.0.0.1:5432

The application tried to connect to a database at 127.0.0.1, and the connection was refused. Inside a container, 127.0.0.1 points to the container itself, not to the host and not to a neighboring container with the database. You need the database service's name on the shared network instead, for example db:5432. This trap was covered in detail in the lesson on volumes and environment variables.

If the log is long, grab the tail or a time window:

docker logs --tail 50 api        # the last 50 lines
docker logs --since 10m api      # everything from the last 10 minutes
docker logs -f api               # and keep following in real time

Logs live as long as the container does. After docker rm they're gone with it. That's why --rm on a service you're debugging gets in the way: it crashes, gets removed, and there's nothing left to look at.

If services are running through compose, the commands are the same, just through docker compose: docker compose ps -a and docker compose logs --tail 50 api.

docker inspect: container state

If the logs are empty, go to docker inspect. It prints a big JSON with everything Docker knows about the container. You can run it with no flags and find what you need by eye, but it's easier to pull out the State section directly. Take a container that ran out of memory: it's given 20 MB, and tail /dev/zero eats it up in a second.

docker run -d --name oom --memory 20m --memory-swap 20m alpine:3.20.3 tail /dev/zero
docker inspect oom --format '{{json .State}}'

{
  "Status": "exited",
  "OOMKilled": true,
  "ExitCode": 137,
  "Error": "",
  "StartedAt": "2026-10-10T02:53:28.131792422Z",
  "FinishedAt": "2026-10-10T02:53:28.352808381Z"
}

The output is shortened and split across lines. Docker prints it as one line, with more fields. The key part is visible right away: code 137 and OOMKilled: true. The container hit its memory limit, and the kernel killed the process. If OOMKilled is false and the code is 137, the process was killed from outside: docker kill or docker stop after its timeout.

You might not have set a limit at all. Then the container grows until the whole machine runs out of memory, and the kernel's OOM killer picks a victim on the host itself. We'll cover the mechanics of limits in the lesson on resources.

The Error field gets filled in when the process never started at all. For the nobash container from the previous section:

docker inspect nobash --format '{{.State.Status}} {{.State.ExitCode}} {{.State.Error}}'

created 127 ... exec: "bash": executable file not found in $PATH: unknown

The container keeps restarting in a loop

It's worse when a container is stuck in a loop. With the restart policy --restart unless-stopped, Docker brings a crashed container back up again. Here's a service that crashes two seconds after starting:

docker run -d --name loop --restart unless-stopped alpine:3.20.3 sh -c 'echo starting; sleep 2; exit 1'

Wait about ten seconds and check:

docker ps --filter name=loop --format '{{.Status}}'

Restarting (1) Less than a second ago

docker inspect loop --format '{{.RestartCount}}'

4

The number in parentheses next to Restarting is the code from the last crash, and RestartCount counts the attempts. The pauses between restarts grow: 100 ms, 200 ms, 400 ms, and keep doubling, up to a minute. That way Docker doesn't let a broken service restart a thousand times a second. If the container stays up for at least 10 seconds after a restart, the pause resets back to 100 ms.

docker events helps you see the timeline of crashes. It's a stream of Docker events: created, started, process died. Without --since it shows only new events, so set a window into the past and filter by the die event:

docker events --since 5m --until 0s --filter container=loop --filter event=die

2026-10-10T05:54:15.090197083+03:00 container die ed4379b6fd15... (exitCode=1, image=alpine:3.20.3, name=loop)
2026-10-10T05:54:18.188732918+03:00 container die ed4379b6fd15... (exitCode=1, image=alpine:3.20.3, name=loop)
2026-10-10T05:54:21.558857836+03:00 container die ed4379b6fd15... (exitCode=1, image=alpine:3.20.3, name=loop)

The output is shortened. --until 0s means "up to the current moment". Without it, the command would hang waiting for new events. The timestamps show the container lives for a couple of seconds and crashes with code 1. So the problem is at application startup, not under load an hour in.

The loop keeps going until you remove the container: docker rm -f loop.

Getting inside an image that won't start

docker exec only works on a running container. If the container crashes a second after starting, you won't get in time.

The way out: run the same image, but swap the entry point for a shell.

docker run --rm -it --entrypoint sh myapp:1.0

Why not docker run -it myapp:1.0 sh? That only replaces CMD. If the image has an ENTRYPOINT set, sh becomes its argument, and the application starts anyway. --entrypoint replaces the entry point itself.

The application doesn't start, so there's nothing to crash. Inside, it's an ordinary check:

ls -la /app                 # is the code there?
cat /app/config.yml         # is it the right config?
env | grep DB_              # did the variables arrive?
which node                  # is the binary there?

Here you're looking at the image's own files. If in production the service gets variables through -e or --env-file, and a config through -v, pass all of that here too. Otherwise you'll be looking at something different from what the application actually sees:

docker run --rm -it --env-file .env -v "$PWD/config.yml:/app/config.yml" --entrypoint sh myapp:1.0

Another trick: pull a file out of a stopped container onto the host. docker cp works even on a crashed one:

docker run --name crash alpine:3.20.3 sh -c 'echo boom > /err.log; exit 1'
docker cp crash:/err.log ./err.log
cat err.log

boom

Handy when the application writes its log to a file instead of stdout.

"no such file or directory", even though the file is right there

This error is confusing: ls shows the file, but Docker says it isn't there. Let's build an image on alpine with a script whose first line is #!/bin/bash:

docker build -t shebang-demo:1 - <<'EOF'
FROM alpine:3.20.3
RUN printf '#!/bin/bash\necho hi\n' > /start.sh && chmod +x /start.sh
CMD ["/start.sh"]
EOF

docker run --name sb shebang-demo:1

exec /start.sh: no such file or directory

The shebang here points to /bin/bash, and alpine doesn't have bash, only /bin/sh. The kernel can't find the interpreter and prints "no such file", even though it's not talking about the script itself.

Check with --entrypoint:

docker run --rm --entrypoint sh shebang-demo:1 -c 'head -1 /start.sh; ls /bin/bash'

#!/bin/bash
ls: /bin/bash: No such file or directory

To fix it, change the shebang to #!/bin/sh, or install bash with apk add bash.

Notice the difference from nobash. When bash sits right in CMD, the container doesn't even start: code 127, status Created, the cause in .State.Error. When bash is only needed by the shebang, the process does manage to start. The container exits with code 1, and the error is in docker logs.

The same text can have another cause too. The script was saved on Windows, and its lines end in \r\n. The kernel looks for the interpreter /bin/sh\r, which doesn't exist. The \r character is invisible to the eye, but cat -A shows it (on alpine and on Linux; locally on macOS use cat -v):

cat -A start.sh

#!/bin/sh^M$
echo hi^M$

^M at the end of the lines is the \r. To fix it, run dos2unix start.sh or set your editor to save files with LF line endings.

exec format error

exec /app/server: exec format error

The file exists and runs, but the kernel doesn't understand its format. Most often the binary was built for a different CPU architecture. For example, the image was built on a Mac with an M-series chip for arm64, but the server runs on amd64. A quick check: rebuild with --platform linux/amd64. We'll cover this in detail at the end of the track.

A second cause: a script with no shebang on its first line. The kernel has nothing to run it with, and the container exits with code 1 and the line exec /start.sh: exec format error in the logs.

Debugging a running container with no shell

Sometimes a service is running but acts strangely: doesn't respond on its port, returns errors. Getting in through docker exec -it web sh doesn't work, because the image has no shell. That's how minimal images are built: distroless contains only the application and the libraries it needs, and scratch is empty altogether.

A neighboring container with tools helps here. Take busybox: it's a tiny image from Docker Hub with sh, ps, wget, netstat, and other basic utilities built in. You can attach it to a service's namespace, and busybox will see the service's processes and network as if it were running inside it.

A plain nginx works well as an example:

docker run -d --name web nginx:1.27.3-alpine
docker run --rm -it --pid=container:web --network=container:web busybox:1.36.1 sh

Inside:

ps                        # nginx's processes
netstat -tlnp             # which ports the service is listening on
wget -qO- localhost:80    # a request to it from inside its own network

PID   USER     TIME  COMMAND
    1 root      0:00 nginx: master process nginx -g daemon off;
   30 101       0:00 nginx: worker process
...
Proto Recv-Q Send-Q Local Address    Foreign Address   State    PID/Program name
tcp        0      0 0.0.0.0:80       0.0.0.0:*         LISTEN   1/nginx: master pro

netstat's flags mean: -t TCP, -l listening sockets only, -n numbers instead of names, -p which process holds the port. If the service runs as a non-root user, the process column might come up empty. Then add the --cap-add SYS_PTRACE flag to busybox.

The --network=container:web flag puts busybox in the service's network namespace, and localhost is now shared between them. --pid=container:web does the same for processes. The service itself isn't touched, and nothing extra is installed into its image.

The trick only works while the service is running. You can't attach to a stopped container. Docker will answer cannot join network of a non running container. If an image with no shell crashes on startup, you're left with docker logs, docker cp for files, and a debug variant of the image. distroless has :debug tags with busybox added: temporarily build the service on one of those and get in through --entrypoint sh.

Order of steps

When a container crashes or won't come up, go through the steps:

1. docker ps -a: status and exit code.
2. docker logs --tail 50 <name>: what the application managed to write.
3. docker inspect <name> --format '{{json .State}}': OOMKilled and Error, if the logs are empty.
4. docker events --since 10m --until 0s --filter container=<name>: how often it crashes.
5. docker run --rm -it --entrypoint sh <image>: look inside an image that won't start.

Most of the time the answer shows up by step two. But with OOM and with a process that never started, the logs are empty, and the third step saves you.

When you're done with the lesson's examples, remove everything you created:

docker rm -f api nobash oom loop sb crash web
docker rmi shebang-demo:1

Common mistakes

• Checking docker ps without -a and deciding the container was never created.
• Treating any 137 as a memory shortage. Check OOMKilled first.
• Putting --rm on a service you're debugging: the logs disappear with the container.
• docker run -it image sh instead of --entrypoint sh: with an ENTRYPOINT set, the application starts anyway.
• Looking for the script itself when it says no such file or directory. Most often there's no interpreter from the shebang, or the file has \r\n line endings.
• Trying to attach busybox to a crashed container. A namespace only exists for a running one.
• Running docker events without --until and then waiting for output that never comes.

What you'll need for the tasks

• docker ps -a, docker logs --tail, docker inspect --format '{{json .State}}'
• the fields .State.ExitCode, .State.OOMKilled, .State.Error, .RestartCount
• docker events --since ... --until 0s --filter ...
• docker run --rm -it --entrypoint sh <image> and docker cp
• cat -A (or cat -v on macOS) to see \r at the end of lines
• docker run --rm -it --pid=container:<name> --network=container:<name> busybox:1.36.1 sh and netstat -tlnp inside
• for the mini-project: docker build, docker push, secrets: in compose from the lessons on registries and secrets