RAID: levels 0, 1, 5, 10

Why this matters

Disks die. On a server with a database or files, one disk failure shouldn't stop the work, and sometimes you just need more speed than a single disk gives. RAID solves both problems. An on-call DevOps engineer needs to be able to read an array's state and tell how much margin it has left: that decides whether to fix it now or it can wait until morning.

What RAID is

RAID (Redundant Array of Independent Disks) is a way to combine several physical disks into one logical volume. There are two goals: get speed higher than a single disk, or survive a disk failure without losing data. Different RAID levels solve these problems differently, and each has its own price in capacity.

On Linux, software RAID is handled by the kernel's md driver, and arrays are assembled and managed with the mdadm utility. A finished array shows up as a device, /dev/md0, /dev/md1 and so on, and you put a filesystem on it like on any regular disk.

How to read /proc/mdstat

The kernel shows the current state of every array in /proc/mdstat:

cat /proc/mdstat

Personalities : [raid1]
md0 : active raid1 sdb1[1] sda1[0]
      976630464 blocks super 1.2 [2/2] [UU]

What matters here:

• md0 : active raid1 — the array's name and its level.
• sdb1[1] sda1[0] — the partitions in the array, with the role number (the disk's position in the array) in brackets.
• [2/2] — how many disks the array needs and how many of them are running.
• [UU] — the state by role, starting from role 0: U means the disk is in place, _ means it's missing.

If a disk drops out, the picture changes. The broken partition gets an (F) (failed) marker, the counter shows [2/1], and a _ shows up in the status string where its role used to be: [U_] or [_U]. An array like that is called degraded: it still works, but its safety margin is already spent in full or in part.

For more detail on a single array, use mdadm --detail /dev/md0 (needs root). The State line of a degraded array will say degraded.

RAID 0: striping

RAID 0 cuts data into blocks (chunks) and spreads them across disks in turn. This is striping. Reads and writes happen on every disk in parallel, so speed goes up. You need at least 2 disks to build one.

There's no fault tolerance at all here. Every file's pieces are scattered across all the disks, so a failure of any single disk loses the entire volume.

mdadm --create /dev/md0 --level=0 --raid-devices=2 /dev/sdb /dev/sdc

The --chunk flag sets the block size. If you don't set it, mdadm uses 512 KiB when creating the array (mdadm(8)).

Usable capacity equals the sum of the disks: two 1 TB disks give you 2 TB. People reach for RAID 0 where they need speed and the data is either disposable or lives somewhere else too.

RAID 1: mirroring

RAID 1 writes identical copies to every disk in the array. This is called a mirror. The volume stays alive as long as at least one disk is alive: out of N disks, N-1 can fail (md(4)). A pair is the usual setup.

The price for reliability is capacity. Out of two 1 TB disks, 1 TB is available, 50% usable capacity. Out of three disks, a third is available.

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc

You can read from any disk, and the driver spreads read requests across all of them, so reads get faster under parallel load. Writes go to every disk at once, so writes never get faster than a single disk.

RAID 5: striping with parity

RAID 5 also lays data out in stripes, but adds a parity block to every stripe. Parity is computed as the XOR of the rest of the blocks in the stripe. Parity blocks are spread across all the disks in turn, not stuck on one. You need at least 3 disks.

XOR gives you a simple property: if you XOR together every block in a stripe, including parity, you get zero. That means any single missing block equals the XOR of all the other blocks in that same stripe, no matter whether it held data or parity. That's how md rebuilds a dropped disk.

An example with a single byte, three disks:

disk A (data)    10110010
disk B (data)    01101100
disk C (parity)  11011110   = A XOR B

Disk B is gone. Compute A XOR C: 10110010 XOR 11011110 = 01101100, exactly B's content.

The array survives one disk failure. Usable capacity is (n-1)/n of the sum: three 1 TB disks give you 2 TB, four give you 3 TB.

mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sdb /dev/sdc /dev/sdd

The dangerous moment here is the recovery window. Once the array is degraded, it already has no margin left: you swap in a disk, and md rebuilds the stripes onto it. A second failure before the rebuild finishes loses the whole volume. The bigger the disks, the longer the rebuild takes, and the longer this window stays open.

When surviving one failure isn't enough, people use RAID 6: it keeps two parity blocks per stripe, and the array survives losing any two disks, at the cost of one more disk's worth of capacity (md(4)).

RAID 10: mirroring plus striping

Classic RAID 10 (also written 1+0) first assembles disks into mirrored pairs, then stripes across the pairs. You need at least 4 disks, and usable capacity is 50%, same as a mirror.

The array survives several failures at once under one condition: no mirrored pair can lose both its disks. One disk dropping out of each of several different pairs still leaves the volume alive. Both disks of the same pair dropping out loses the volume.

mdadm --create /dev/md0 --level=10 --raid-devices=4 \
  /dev/sdb /dev/sdc /dev/sdd /dev/sde

In md, the raid10 level is its own separate driver: it keeps a set number of copies of every block (usually 2) and can run on as few as 2 or 3 disks, as long as there are at least as many disks as copies (md(4)). On 4 disks with two copies, it behaves like a classic 1+0. In /proc/mdstat you can see this as a 2 near-copies line.

People reach for this kind of array under busy databases: speed from striping and resilience from mirroring, at the cost of half the capacity.

Comparing levels

Level	Minimum disks	Usable capacity	Survives
RAID 0	2	100%	0 disks
RAID 1	2	1/n (50% for a pair)	n-1 disks
RAID 5	3	(n-1)/n	1 disk
RAID 6	4	(n-2)/n	any 2 disks
RAID 10	4	50%	1 disk from each pair

RAID is not a backup

The big beginner trap: treating RAID as a replacement for backups. RAID protects against one class of disaster, a dead disk. Against an accidental rm -rf, a bug in a script, or ransomware it's useless: the array faithfully mirrors or stripes data that's already been destroyed.

rm -rf /var/data/*

A mirror will happily run this command on both disks. A separate backup on different media or in a different place stays mandatory.

Common mistakes

• Not watching the array. A degraded RAID keeps working like nothing happened, and people find out about the dropped disk only when the second one fails. You need monitoring: mdadm --monitor sending email, or a check of /proc/mdstat in your monitoring system.
• Assuming a RAID 5 with one dropped disk still "survives a failure". The margin is already spent: any next failure before the rebuild finishes loses the volume.
• Mixing up [2/2] and [UU]. The first number in the counter is how many disks are needed, the second is how many are running. [UU] shows exactly which role is empty.
• Building RAID 0 under data you'd mind losing. One disk failure out of any number wipes everything.
• Treating RAID as a backup.

What you'll need for the tasks

• The array's line in /proc/mdstat: name, level, partitions with a role number, the (F) marker on a dropped one.
• [n/m]: how many disks are needed and how many are running; [UU_U]: which role is empty.
• How many failures each level survives (table above) and how many already happened.
• A RAID 5 missing block equals the XOR of all the other blocks in the same stripe.
• XOR of two bytes in Python: a ^ b; reading a file as bytes: open(path, "rb").read().

Next up is a short quiz on the filesystem track.