initramfs: a temporary root in memory

A server updates its kernel, reboots, and instead of a login prompt shows (initramfs) or a Kernel panic. The culprit is usually neither the disk nor the kernel, but a small archive that loads together with the kernel. This lesson covers why it exists, what's inside it, and how to open it, check it, and repack it.

The chicken-and-egg problem at boot

By this point the boot chain has already passed through several hands: firmware hands control to the bootloader, the bootloader loads the kernel (the kernel is the operating system's central program, managing memory, processes, and hardware). Next the kernel needs to mount the real root partition to start the first process. And that's where things get stuck.

To read the root partition, the kernel needs drivers: a disk controller module, RAID code, LVM support (Logical Volume Manager, a layer that combines physical partitions into flexible logical volumes). And these modules live on that very partition, which isn't mounted yet. You can't read a driver without a driver.

You could build everything into the kernel statically, but then the kernel would bloat with drivers for every piece of hardware out there. Distributions do it differently: one kernel for everyone, and the modules a specific machine needs travel alongside it in initramfs.

What initramfs is

initramfs is a compressed cpio archive (cpio is an old archive format; the kernel uses its newc variant), which the bootloader places in memory next to the kernel. In GRUB, the initrd line in the menu entry handles this. The kernel unpacks the archive into rootfs. rootfs is a filesystem right in RAM: ramfs, or tmpfs if the kernel is built with CONFIG_TMPFS.

Inside the archive sits a minimal set: a few programs (often just busybox), the needed kernel modules, and an init script at the root. The script knows how to find and mount the real root.

After unpacking, the kernel looks for a file /init in rootfs and runs it as the first process. If /init is missing or can't run, the kernel tries to mount the root itself, using the root= parameter and only the drivers built into the kernel. On a distribution kernel this usually ends in a panic.

How to look inside

On Debian and Ubuntu, lsinitramfs from the initramfs-tools package shows the file list:

lsinitramfs /boot/initrd.img-$(uname -r) | head

Output looks roughly like this (trimmed):

.
kernel
kernel/x86/microcode/GenuineIntel.bin
...
init
scripts/local
usr/lib/modules/6.8.0-45-generic/kernel/drivers/md/dm-mod.ko
...

You can see the module dm-mod.ko here. That's the device-mapper driver, which LVM runs on. Without it, you can't mount an LVM root.

On Fedora and RHEL, dracut builds the image, and lsinitrd shows its contents:

lsinitrd /boot/initramfs-$(uname -r).img | head

How to unpack it by hand

The format is open, but a modern image has two quirks that often break the old recipe zcat initrd.img | cpio -idmv.

First: compression. initramfs-tools defaults to COMPRESS=zstd, not gzip, and zcat can't read that. Check the compression with the file command.

Second: concatenation. The start of the image often carries an uncompressed cpio with CPU microcode (kernel/x86/microcode/...), followed by the main, compressed one. The kernel reads them in sequence. Plain cpio will only unpack the first chunk, and zcat trips on the very first byte.

That's why on Debian and Ubuntu you unpack it like this:

unmkinitramfs /boot/initrd.img-$(uname -r) /tmp/initrd

/tmp/initrd will end up holding the image's tree. Recent versions of initramfs-tools merge the microcode and the main archive into one tree; older ones sometimes split them into separate subdirectories. On Fedora, lsinitrd --unpack unpacks into the current directory.

The manual cpio recipe works for a simple, single-chunk archive compressed with gzip. That's exactly what the labs build:

mkdir /tmp/initrd
cd /tmp/initrd
zcat /path/to/initrd.img | cpio -idm

Here -i means unpack, -d create directories, -m keep file modification times. After unpacking you'll see a small directory tree with its own bin, lib, scripts, and an init file at the root.

How to repack it

The kernel docs describe the reverse operation. It's important to run it from the root of the unpacked tree:

cd /tmp/initrd
find . | cpio -o -H newc | gzip > /tmp/initrd.img

-o means create an archive, -H newc sets the format the kernel understands. find . prints paths relative to the current directory, and that's exactly how they end up in the archive.

cpio preserves file permissions as they are. init needs the execute bit set, or the kernel won't run it.

In practice you rarely build it by hand: the image gets rebuilt with update-initramfs -u (Debian, Ubuntu) or dracut -f (Fedora, RHEL). Manual building matters when you're fixing an image from a live disk or figuring out exactly what ended up in it.

Switching to the real root

The init script inside initramfs loads the needed modules, finds the root partition (by label, UUID, or path), and mounts it, usually at /root. From there, the temporary in-memory root gives way to the real partition on disk. Simplified, the end of the script looks like this:

mount /dev/mapper/vg0-root /root
exec switch_root /root /sbin/init

switch_root deletes the contents of rootfs to free up memory, makes /root the new root, and runs /sbin/init there. That's the first real process with PID 1, usually systemd. Through exec it inherits the same PID that init from initramfs had.

The old initrd mechanism (a RAM disk with an ext2 image instead of a cpio archive) switched over via pivot_root. Recent kernel docs call pivot_root the preferred way for initramfs too, but you still see switch_root in Debian, Ubuntu, and Fedora. initramfs has existed in every kernel since 2.6.

Common mistakes

"initramfs is the root filesystem on disk." No. It's a separate, temporary in-memory image. It lives for seconds and exists only to mount the real root. The real / sits on disk and gets attached later, via switch_root.

"You can delete it, it's not needed." If you delete the image file, on the next boot the kernel won't find the disk drivers and will stop before it even mounts the root. The message at the end will look like this:

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

"The root isn't showing up, so the disk must be broken." If init from initramfs started but couldn't find the device from root=, initramfs-tools waits and then gives up with this message:

Gave up waiting for root device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/mapper/vg0-root does not exist.  Dropping to a shell!

After that, a shell opens with the (initramfs) prompt. The disk is often fine, it's just that the image is missing the module needed to reach it. Compare the contents of the new image with the old one that used to boot.

"I'll just unpack it with zcat | cpio." On Debian 12 and Ubuntu 24.04, the image is zstd-compressed by default, and microcode is often glued to the front. Use unmkinitramfs or lsinitrd instead.

What to remember

initramfs is a compressed cpio archive that the kernel unpacks into rootfs in RAM and runs /init from.

It solves the chicken-and-egg problem: it holds the drivers (disk, RAID, LVM) needed to mount the real root.

lsinitramfs and lsinitrd show the contents, unmkinitramfs unpacks it. A simple gzip archive opens with zcat | cpio -idm and repacks with find . | cpio -o -H newc | gzip.

After mounting the real root, init runs switch_root and starts PID 1.

Don't delete the image: without it you'll get a kernel panic when mounting the root.

The next lesson covers what the first process does: systemd, unit files, and starting services in parallel.

What you'll need for the tasks

• zcat image | cpio -t prints the archive's file list without unpacking it.
• zcat image | cpio -idm unpacks into the current directory.
• find . | cpio -o -H newc | gzip > image builds an archive from the current directory.
• sort, diff, and comm compare two lists; the kernel version number in the paths gets in the way of comparing, you can strip it with sed.
• ls -l shows permissions, chmod +x sets the execute bit.
• The root= parameter in the kernel command line tells you which device the root lives on.