Snapshots: LVM, ZFS, and disks in the cloud

A volume snapshot takes a second to create and minutes to roll back, so people take one before a schema migration, a system upgrade, or a risky release. But it has two catches. It lives right next to the original, and a snapshot of a running database comes back "as if after a power outage". This lesson covers how a snapshot works, where it helps, and why it isn't a backup by itself.

Why it's instant

A snapshot of a terabyte volume is created in a second, and that's the first thing that looks suspicious. No copying happens: the system records the current state of the blocks and from there works by one of two mechanisms.

The first, copy-on-write: when a write lands on a block that's part of the snapshot, the old contents move out to a separate area first, and only then is the new data written. Classic LVM snapshots and Windows shadow copies work this way.

The second, redirect-on-write: the block is never overwritten at all, new data lands in free space, and the old block keeps living as long as the snapshot still references it. This is how ZFS and Btrfs work.

In both cases, the snapshot doesn't take up space right away, only as changes accumulate. The more you change after taking it, the heavier it gets.

The practical consequence: a snapshot forgotten on an active database keeps growing and eventually runs into whatever space was set aside for it.

LVM

In LVM, a snapshot is taken at the logical volume level:

lvcreate --size 10G --snapshot --name db-snap /dev/vg0/db

The size here is space for changes, not for a copy of the data. And it's carved out of the volume group immediately: ten gigabytes become unavailable the moment the snapshot is created, even while it's still empty.

Run out of the reserved space, and a classic snapshot becomes invalid: the kernel disables it and returns errors on reads, and lvs shows it in state I (invalid). Thin snapshots don't reserve size, space comes out of a shared pool. Then it's the pool itself that overflows: by default, writes to thin volumes queue for up to 60 seconds and then start returning I/O errors. At that point every volume in the pool suffers, not just the snapshot.

From there you can mount the snapshot and calmly copy the data while the main volume keeps running. This is the classic move: snapshot for a minute, copy from it, delete the snapshot.

ZFS and Btrfs

In copy-on-write filesystems, snapshots feel more natural, because the mechanism is always running rather than switched on specially for the occasion:

zfs snapshot tank/data@2026-08-23
zfs list -t snapshot

The upside is that these snapshots can be streamed to another server:

zfs send tank/data@2026-08-23 | ssh backup-host zfs recv backup/data

The receiving dataset must either not exist, or accept the stream with the -F flag, otherwise zfs recv refuses to write over an existing one.

Now this starts to look like an actual backup: the copy leaves for another machine. A snapshot sitting in the same pool by itself isn't a backup.

Disk snapshots in the cloud

Cloud providers usually store a disk snapshot in object storage, separate from the disk itself. For example, EBS snapshots on AWS live in Amazon S3 and are replicated across all availability zones in the region. That's notably better than a local snapshot: the disk dying doesn't take the copy with it.

Cloud snapshots are also incremental: each next one only stores the blocks changed since the last. So deleting one snapshot doesn't always shrink the bill: the blocks it shares with its neighbors stay billed to them.

Two general risks remain. First: snapshots live in the same account, and an administrator with the right permissions (or whoever hijacked their access) deletes them with a single command, unless a separate lock like EBS snapshot lock is enabled. Second: they're tied to the provider and become unavailable along with everything else if there's a problem with the account.

So cloud snapshots are good as a fast rollback and the first layer, but they don't satisfy the 3-2-1 rule.

Consistency: the main catch

A volume snapshot captures the disk exactly as it was at that moment, including writes that hadn't finished yet.

A crash-consistent snapshot looks like the state after a sudden power loss. The filesystem replays its journal on mount, the database replays its transaction log on startup. This usually works, but only if both the log and the data made it into the snapshot, and at the same moment.

An application-consistent snapshot is made differently: before taking it, the application is asked to bring its data into a consistent state, flush its buffers to disk, and hold off on writes for an instant.

What a crash-consistent snapshot looks like for Postgres. If you bring up a database from such a snapshot, the log will have these lines:

LOG:  database system was not properly shut down; automatic recovery in progress
LOG:  redo starts at 0/14EFD48
...
LOG:  database system is ready to accept connections

The database decided the previous process crashed, and replayed the WAL (write-ahead log) from the last checkpoint. The PostgreSQL documentation flatly calls this normal and asks for only one thing: include the WAL files in the snapshot.

If all the data lives on one volume and the snapshot is atomic, nothing else is needed. It gets harder when the database is spread across several filesystems: the WAL on a separate disk, tablespaces on a third one. Then the snapshots of all the volumes have to be taken at the same exact moment, which isn't always achievable.

For this case Postgres has a low-level interface:

SELECT pg_backup_start('snapshot');
-- take the volume snapshot here
SELECT * FROM pg_backup_stop(wait_for_archive => true);

There are three catches here. The connection has to stay alive between the calls: if it drops, the backup counts as interrupted. pg_backup_stop() returns a row with three fields: the second one needs to be saved next to the copy in a backup_label file, the third in a tablespace_map file (if it isn't empty). Without them the copy won't come up. And the database doesn't freeze writes during this: consistency comes not from a pause but from the WAL, which you'll need to apply to the snapshot from the archive.

These names date from PostgreSQL 15; before that the functions were pg_start_backup and pg_stop_backup, and older articles still use those names.

On Windows, VSS handles the same job, but not by itself: consistency comes from a writer inside the application, which finishes transactions and flushes caches before the snapshot. Cloud providers have agents and pre-scripts for this.

A snapshot on your laptop: freeze and copy

LVM and ZFS can't be brought up inside a regular container without privileged mode, so in the labs the snapshot is done by hand. The idea is the same as at the filesystem level: freeze writes, capture the contents, let go.

docker pause freezes all of a container's processes through the cgroup freezer. While the container is paused, none of its processes can write a single byte, and the volume's contents can be copied by a neighboring container. The result is a crash-consistent snapshot: everything the database already handed to the kernel is in the copy, and its in-memory buffers are gone, same as after a power outage.

docker pause <container>
# copy the volume
docker unpause <container>

The difference from a real snapshot is speed. Copying a 40 MB volume on Docker Desktop took about 15 seconds, and the database didn't answer the whole time. LVM or ZFS hold that pause for a fraction of a second, because they don't copy anything.

When a snapshot makes sense

Before a schema migration, before a system upgrade, before rolling out a risky release. It costs seconds, rolls back in minutes, and covers "we shipped it and broke it" perfectly.

As the only line of data protection, a snapshot doesn't hold up, for the reasons covered in the first lesson of this track. It lives next to the original, gets deleted with the same permissions, and depends on the same provider.

Common mistakes

• Forgetting an LVM snapshot on an active volume. A classic snapshot fills up and becomes invalid, a thin one fills the pool and stops writes to every volume in it.
• Treating a snapshot in the same pool or account as a backup. The pool dies or the account gets hijacked, and the snapshots go with the data.
• Snapshotting the volumes of a spread-out database separately: the data at one moment, the WAL at another. Snapshots have to be simultaneous.
• Getting scared by the not properly shut down line when bringing up a database from a snapshot. For a crash-consistent snapshot that's expected, the database just replays the WAL on its own.
• Copying a running database's directory with cp or rsync without a freeze and calling it a snapshot. The files get copied at different moments, and the copy may end up without a consistent state at all.

What you'll need for the tasks

• docker pause <container> and docker unpause <container>: freeze and release all of a container's processes;
• docker volume create --label <key>=<value> <volume>: create a volume with a label;
• docker run --rm -v <volume>:/from:ro -v <volume2>:/to alpine:3.20.3 cp -a /from/. /to/: copy the contents of one volume into another, keeping owners and permissions;
• docker logs <container>: the database log, it shows whether recovery happened;
• docker inspect -f '{{json .Mounts}}' <container>: which volumes are mounted where;
• docker exec <container> psql -U postgres -Atc '<query>': run SQL inside a Postgres container.

Next up is RAID: what it actually protects against, and why rebuilding a large array is an event of its own.