The bootloader and GRUB

A server won't come up after a kernel update: the screen shows the GRUB menu or No bootable device. To make sense of a failure like this, you need to know who's working at this stage: firmware is already done, the kernel hasn't started yet. Between them sits the bootloader. This lesson covers what it does, where it lives on disk, and where it gets its settings.

What the bootloader does

The bootloader is a program that finds the kernel on disk, loads it into memory and hands it control. Firmware can't do much: check the hardware and run the first code from the disk. It doesn't need to read a filesystem with a kernel on it or understand the kernel's format; that job goes to the bootloader.

The startup chain is always the same: firmware -> bootloader -> kernel -> initramfs. Each link does its piece of work and hands off to the next. initramfs is a temporary root that the kernel unpacks into memory (into rootfs, which is tmpfs if the kernel is built with CONFIG_TMPFS), so it can later mount the real root partition.

Besides the kernel itself, the bootloader passes it parameters: the kernel command line, which says where to find the root, how to mount it, and what to show on screen.

root=/dev/sda2 ro quiet

Here root= says which partition to mount as root, ro tells it to mount that partition read-only at first, quiet removes kernel log lines from the screen during boot. On modern distributions, root=UUID=... is usually used instead of a device name: the name /dev/sda2 can change if you rearrange disks, but a filesystem's UUID doesn't.

The parameters the running system booted with are in /proc/cmdline:

cat /proc/cmdline

BOOT_IMAGE=/vmlinuz-6.8.0-45-generic root=UUID=0f3c2a1e-... ro quiet splash

BOOT_IMAGE= is added by GRUB: it's the path to the kernel file it loaded. Your output will be different.

GRUB

GRUB (GRand Unified Bootloader) is the bootloader on most Linux systems. At startup it shows a menu: which kernel or which system to start. GRUB reads the menu and all its settings from the file grub.cfg.

ls /boot/grub/

fonts  grub.cfg  grubenv  i386-pc  locale  unicode.pf2

This is what the directory looks like on Ubuntu with BIOS. The i386-pc directory holds GRUB's modules for BIOS; on UEFI machines it's x86_64-efi instead. On Fedora and RHEL the directory is called /boot/grub2/.

Nobody edits grub.cfg by hand. A tool builds it from templates in /etc/grub.d/ and settings in /etc/default/grub, and manual edits disappear the next time it's rebuilt. After changing the settings, rebuild the config:

sudo update-grub

That's Debian and Ubuntu, where update-grub calls grub-mkconfig -o /boot/grub/grub.cfg. On Fedora and RHEL:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Kernel parameters for all menu entries are set in /etc/default/grub, in the GRUB_CMDLINE_LINUX_DEFAULT line (Debian, Ubuntu) or GRUB_CMDLINE_LINUX. RHEL also has a grubby tool for this.

GRUB can do chainloading: hand control to another bootloader instead of loading a kernel itself. This is how Windows gets started on a computer that also has Linux: GRUB doesn't understand how Windows boots and hands control to Windows' own bootloader.

menuentry "Windows" {
    chainloader (hd0,1)+1
}

This is a menu entry for BIOS: (hd0,1) is the first partition of the first disk (GRUB numbers disks from zero, partitions from one), +1 means "the first sector of this partition". On UEFI, instead of a sector you point to the Windows bootloader's file on the EFI partition.

Where the primary code physically lives

On BIOS systems, firmware reads the disk's first sector, the MBR (Master Boot Record), and runs the code in it. The sector is 512 bytes, and 446 of those are left for code.

sudo dd if=/dev/sda bs=512 count=1 | wc -c

1+0 records in
1+0 records out
512 bytes copied, ...
512

The records in/out lines are printed by dd itself on stderr, wc -c counts the 512 bytes of the MBR. On a laptop with NVMe, the disk is called /dev/nvme0n1, not /dev/sda.

The bootloader doesn't fit into 446 bytes, so GRUB loads itself in stages. The MBR holds boot.img, exactly 512 bytes with the partition table. Its code can do one thing: read the first sector of core.img from disk. core.img loads the rest of itself, already understands filesystems, finds /boot/grub, loads modules, reads grub.cfg and draws the menu. On an MBR disk, core.img usually sits in the gap between the MBR and the first partition.

UEFI works differently. The bootloader is an ordinary file on the EFI partition (ESP, a partition with a FAT filesystem), for example /boot/efi/EFI/ubuntu/grubx64.efi, and firmware runs it directly. No MBR code is needed here. A Linux kernel can be built with an EFI stub, and then firmware runs the kernel itself as an executable, with no separate bootloader at all.

What is inside the MBR

The first sector splits into three parts:

Bytes	What's there
0-445	primary bootloader code (bytes 440-443 are usually taken by a disk signature)
446-509	partition table: 4 entries of 16 bytes each
510-511	signature 55 aa

The signature is how BIOS knows the sector is bootable. If those bytes are anything else, BIOS won't run the code in the sector and tries the next device or prints an error. Documentation usually writes the signature as the 16-bit number 0xAA55: x86 stores numbers with the least-significant byte first, so in the file the bytes come as 55 aa.

In a partition table entry, the fifth byte (offset 4 from the start of the entry) sets the partition type: 83 Linux, 82 Linux swap, 07 NTFS, ee a GPT protective entry. An empty entry is all zeros. The entry's first byte is the active-partition flag 80; DOS and Windows bootloader code reads it, GRUB doesn't look at it. The entry's last 8 bytes hold the partition's start and its length in sectors.

Bootloader code almost always has text error messages. GRUB's are GRUB, Geom, Hard Disk, Read, Error. That's why it's useful to read the first sector as text: the lines show whose bootloader it is. od -A d -t x1 shows bytes with addresses in decimal, -j skips the start, -N limits the length:

sudo dd if=/dev/sda bs=512 count=1 2>/dev/null | od -A d -t x1 -j 496

0000496    00  00  00  00  00  00  00  00  00  00  00  00  00  00  55  aa
0000512

Common mistakes

"GRUB is part of the kernel." No, it's a separate program. GRUB loads the kernel, hands it control, and doesn't run after that. The kernel reuses the memory it occupied like any other memory.

"The bootloader is one file." Also no. GRUB is made of several parts: boot.img in the MBR (or grubx64.efi on the EFI partition), core.img, modules in /boot/grub, the config grub.cfg. They load one after another.

"I'll just edit grub.cfg and be done." That edit works until the next update-grub, and it runs on its own with every kernel update. Change /etc/default/grub or the files in /etc/grub.d/ instead, then rebuild the config.

Once the kernel has control and brought up initramfs, the first user-space process starts, usually systemd.

What to remember

The bootloader finds the kernel on disk, loads it into memory and hands over control along with parameters (the kernel command line). The chain: firmware -> bootloader -> kernel -> initramfs.

GRUB reads grub.cfg, shows a menu, loads itself from disk in stages, and can chainload to another bootloader. Settings are edited in /etc/default/grub, and grub.cfg gets rebuilt. Kernel parameters like root=, ro, quiet show up in /proc/cmdline.

The MBR is 512 bytes: 446 for code, 64 for the partition table, 2 for the 55 aa signature. On UEFI the bootloader is a file on the EFI partition, and with an EFI stub the kernel loads with no separate bootloader at all.

The next lesson covers initramfs: why the kernel needs a temporary root in memory, and how it uses it to mount the real one.

What you'll need for the tasks

• The MBR layout: code in bytes 0-445, the partition table in 446-509, the signature in 510-511.
• A partition table entry: 16 bytes, the partition type in the fifth byte, an empty entry is all zeros.
• head -c N file takes the first N bytes, grep -a searches for text in binary data.
• od -A d -t x1 -j <skip> -N <how many> file shows bytes in hex.
• dd: if=, of=, bs=, count=, seek=. What dd does to the rest of the file after writing, you'll find out in the second lab.