tmpfs: a filesystem in memory

Why this matters

On any server running systemd, some directories live not on disk but in memory: /run, /dev/shm, sometimes /tmp. In Docker, directories like that are created with the --tmpfs flag. If you don't know which directories are in memory, it's easy to lose data after a reboot or to get No space left on device on a server whose disk is nearly empty.

What tmpfs is

tmpfs is a filesystem that keeps its files in memory instead of on disk. A directory looks like any other, you write to it and read from it with the usual commands, but there isn't a single block of it on any storage device.

That gives it its key property: the contents only live until the next reboot. After a shutdown or reboot, everything that was in tmpfs is gone, and there's nothing to recover: nothing was ever written to disk.

echo test > /run/mydata
cat /run/mydata

test

After a reboot, the file /run/mydata is gone. Writing to /run needs root.

In a container, the same thing happens on stop: a tmpfs mounted with --tmpfs is removed along with its files when the container stops (Docker documentation).

Memory is taken as you write

A common mix-up: people call tmpfs a "fixed-size ramdisk". It isn't. A ramdisk is a fixed-size block device carved out of memory. You put a regular filesystem on it, the way you would on a disk. tmpfs works differently: files sit right in the kernel's internal caches, and the filesystem grows and shrinks with its contents (kernel documentation).

So an empty mount point with a 512 MB limit doesn't take 512 MB of memory. Memory is only used for files actually written, and never more than the limit.

df -h /run

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.6G  1.3M  1.6G   1% /run

Here Size is the limit, Used is what's taken right now. If no limit is set at mount time, the kernel sets it to half of physical memory. Docker's --tmpfs without size does the same: up to half the host's memory.

Memory used by tmpfs also shows up in overall statistics: in /proc/meminfo it's counted in the Shmem line, and in free -h output it's in the shared column. Under memory pressure, the kernel can push rarely used tmpfs pages out to swap (a partition or a swap file on disk).

What happens when you hit the limit

Once files in tmpfs grow to Size, writes fail with the same error as a full disk:

dd: error writing '/run/big.bin': No space left on device

Meanwhile df -h / can show dozens of free gigabytes. Free space on the root partition doesn't help: every mount point has its own limit. So when you see No space left on device, check df -h for the specific directory you were writing to.

An unfinished file stays in tmpfs and holds memory until it's deleted. Delete it, and Used drops right away.

Where you'll find tmpfs

A filter by type shows every mounted tmpfs:

mount | grep tmpfs

tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=1600000k)
...

findmnt -t tmpfs is more convenient: same information as a table. And findmnt -T <path> answers the question of which filesystem a specific directory or file lives on.

Typical mount points:

Mount point	What it's for
/run	service state for the current boot: pid files, sockets
/dev/shm	shared memory between processes
/run/user/<uid>	a specific user's session files

/run holds things that only make sense until the next reboot: daemon pid files, unix sockets, transient state. Services recreate it all after boot.

/dev/shm holds POSIX shared memory objects. Several processes open the same object and work on shared data without passing it between each other. In a Docker container, /dev/shm is limited to 64 MB by default, the --shm-size flag changes that.

/tmp: it depends on the distro

Another misconception: "/tmp is always tmpfs". In fact, that's up to the distribution. Debian, for example, has kept /tmp on tmpfs since version 13. On other systems, /tmp is a plain directory on the root partition.

Check your own system:

findmnt /tmp

If the command prints a line with FSTYPE set to tmpfs, /tmp is in memory. If there's no output, /tmp isn't mounted separately and lives on the root partition.

The difference matters. If /tmp is on tmpfs, a large temporary file eats memory and can hit the limit. If /tmp is on disk, the file takes up partition space instead of memory. Either way, you shouldn't store anything there for long: systemd-based distributions usually clean /tmp on boot and remove old files by age. For temporary files that need to survive a reboot, there's /var/tmp: it's usually left alone on boot (Linux File System Hierarchy).

Common mistakes

"I'll put it in /run, it's reliable." Only put things in /run and /dev/shm that you don't mind losing on reboot. Data you'll need tomorrow goes to disk.

"There's plenty of space, df shows gigabytes." df -h with no arguments shows every mount point, and the eye tends to catch the root partition. When you see No space left on device, you need df -h <directory>: if it shows tmpfs and Use% at 100, you've hit the tmpfs limit, not the disk.

"I killed the process, the memory came back." tmpfs memory is held by files, not by a process. As long as a file sits in tmpfs, the memory stays taken, even if the process that wrote it finished long ago.

"tmpfs without a limit is convenient." tmpfs without a limit can take all of the machine's memory. The kernel documentation warns about this directly: a tmpfs that's too large can hang the system.

What you'll need for the tasks

• findmnt -T <path>: which filesystem a directory is on.
• findmnt -t tmpfs or mount | grep tmpfs: every tmpfs in the system.
• df -h <directory>: the limit (Size) and what's used (Used) for a specific mount point.
• du -sh <directory>: how much space the files inside actually take.
• tmpfs contents are gone after a reboot, or after a container stops.
• No space left on device on tmpfs means you hit its limit, not the disk.

In the next lesson we'll look at disk partitions and swap.