Install Debian Bookworm on a Software RAID and EFI
Posted by
ads
on
Friday, 2024-07-19 Posted in [Hardware][Linux][Operating-Systems][Software]
Moving my systems away from Ubuntu, I also have to reinstall a server at home. This system has two equal disks for a RAID system, and my plan is to install Debian on it. Turns out, setting up a Software RAID on Debian and booting from EFI is not a straightforward approach. This blog posting describes the steps.
The “hardware”
For my use cases at home I’m using a server I bought from Thomas Krenn, it has a Intel(R) Xeon(R) E-2224 CPU @ 3.40GHz CPU and 32 GB RAM. It also has two INTEL SSDSC2KB48 SSD disks with 512 GB each. The disks are wiped to remove the previously installed Ubuntu:
1
for i in a b;do wipefs -a /dev/sd$i; dd if=/dev/zero of=/dev/sd$icount=100000;done
This ensures that the Debian installer doesn’t have to deal with existing configurations. I want to again install a SoftwareRAID on these two disks, like I previously had for Ubuntu.
First approach
The Debian disk partition tool only supports manually setting up a RAID. And the “guided” tool does not recognize when a disk/partition on a Software RAID is selected. It happily creates all the necessary partitions, including the UEFI (Unified Extensible Firmware Interface, EFI) partition, on the RAID device.
Guided installer did happily install ESP, LVM and partitions on the one RAIDed partition
Of course this fails in a spectacular fashion!
Install the GRUB boot loader: Unable to install GRUB in dummy - Executing 'grub-install dummy' failed. This is a fatal error.
Install the GRUB boot loader: Installation step failed - An installation step failed. You can try to run the failing item again from the menu, or skip it and choose something else. The failing step is: install the GRUB boot loader
Figuring out the correct RAID setup
What I need to do on the first disk is:
Manually create the boot block partition, mark it as biosgrub (size: 1 MB)
Manually create the EFI partition, mark it as EFI (size: 538 MB, same as the guided installer created)
Manually create the /boot partition, mark it as raid (size: 500 MB)
Manually create the / (root) partition, mark it as raid (remaining size minus 1.1 GB for swap in my case)
Manually create the swap partition, mark it as raid (remaining size)
Looks like this:
Repeat the steps for the second disk.
This results in two equally configured disks. After the partitions are created, configure three (in my case) RAID devices:
md0: /dev/sda3 + /dev/sdb3
md1: /dev/sda4 + /dev/sdb4
md2: /dev/sda5 + /dev/sdb5
The RAID devices after creating the devices:
Click on each RAID device, and select what this device is used for:
md0 will be /boot with the 500 MB partitions
md1 will be / with the large 478 GB partitions
md2 will be swap with the 1.1 GB partitions
From here, continue the installation. It works from here on.
After the installation
Once the installation is completed, and the system has booted into the installed system, a few additional steps are necessary.
First keep an eye on the RAID sync process:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cat /proc/mdstat
Personalities : [raid1][linear][multipath][raid0][raid6][raid5][raid4][raid10]md2 : active (auto-read-only) raid1 sda5[1] sdb5[0]1037312 blocks super 1.2 [2/2][UU]resync=PENDING
md1 : active raid1 sdb4[0] sda4[1]466664448 blocks super 1.2 [2/2][UU][>....................]resync= 1.6% (7880448/466664448)finish=37.0min speed=206184K/sec
bitmap: 4/4 pages [16KB], 65536KB chunk
md0 : active raid1 sdb3[0] sda3[1]487424 blocks super 1.2 [2/2][UU]resync=DELAYED
unused devices: <none>
It will take a while, but eventually all RAID devices must be synced.
In my case, the swap partition is resync=PENDING, and in auto-read-only mode. This is initially delayed until the partition receives the first writes. Can be forced with:
1
mdadm --readwrite /dev/md2
1
2
3
4
5
6
cat /proc/mdstat
Personalities : [raid1][linear][multipath][raid0][raid6][raid5][raid4][raid10]md2 : active raid1 sda5[0] sdb5[1]1037312 blocks super 1.2 [2/2][UU][===>.................]resync= 19.4% (201984/1037312)finish=0.0min speed=201984K/sec
Copy the EFI files to the second disk
Currently the EFI files are only installed on the first disk (/dev/sda2 or /dev/sdb2, depending what the installer choose). In order to be able to boot from the second disk, the files need to be copied from /dev/sda2 to /dev/sdb2 or vice versa. The best place to handle this is a grub hook, which is executed whenever grub updates.
Place the following file in /etc/grub.d/90_copy_to_boot_efi2, and replace $raid_disk_first and $raid_disk_second with the device names for your primary and secondary disk.
#!/bin/bash
set -e # abort on errorset -u # abort on variable not set#set -x # trace execution# one or both devices can already be mounted, potentially with a label and not with the device namedevice_first="/dev/{{ raid_disk_first }}"device_second="/dev/{{ raid_disk_second }}"# the findmnt will fail when the device is not mounted, that is expectedset +e
mountpoint_first=`/usr/bin/findmnt --first-only --noheadings --output=TARGET ${device_first}`mountpoint_second=`/usr/bin/findmnt --first-only --noheadings --output=TARGET ${device_second}`set -e
if[ -z "${mountpoint_first}"];then# not mounted, create the mount pointpath_first="/efi-first" /usr/bin/mkdir "${path_first}"echo"No existing mount point found for ${device_first}"echo" Mounting on ${path_first}"# and mount the decive /usr/bin/mount "${device_first}""${path_first}"else# already mounted, use the existing mount pointpath_first="${mountpoint_first}"echo"Found existing mount point ${mountpoint_first} for device ${device_first}"fiif[ -z "${mountpoint_second}"];then# not mounted, create the mount pointpath_second="/efi-second" /usr/bin/mkdir "${path_second}"echo"No existing mount point found for ${device_second}"echo" Mounting on ${path_second}"# and mount the decive /usr/bin/mount "${device_second}""${path_second}"else# already mounted, use the existing mount pointpath_second="${mountpoint_second}"echo"Found existing mount point ${mountpoint_second} for device ${device_second}"fiif[ ! -d "${path_first}/EFI" -a -d "${path_second}/EFI"];then# there's an "EFI" directory in the second mount point,# but not in the first one - this is wrong, and will damage# the installation once rsync runsecho"There is no 'EFI' directory in the first (source) mount point!"echo"But there is an 'EFI' directory in the second (destination) mount point!"echo"This will not work, possibly switch the devices!"echo"Source: ${device_first} (mounted on: ${path_first})"echo"Destination: ${device_second} (mounted on: ${path_second})"exit1fi#echo "rsync --dry-run --verbose --times --recursive --delete \"${path_first}/\" \"${path_second}/\""#rsync --dry-run --verbose --times --recursive --delete "${path_first}/" "${path_second}/"rsync --verbose --times --recursive --delete "${path_first}/""${path_second}/"if[ -z "${mountpoint_first}"];then# unmount the device /usr/bin/umount "${path_first}"# remove the directory /usr/bin/rmdir "${path_first}"fiif[ -z "${mountpoint_second}"];then# unmount the device /usr/bin/umount "${path_second}"# remove the directory /usr/bin/rmdir "${path_second}"fiexit0
The {{ }} are for Ansible, I’m deploying this script using a Playbook:
Every time something is changed on the original EFI partition, the script will mount the other partition and sync the content.
Lessons learned
Don’t use the guided installer. It happily - and wrongly - accepts a partition on a RAID, and creates additional partitions and even an LVM on that one RAID partition. All partitions must be created manually.
When just dding the beginning of the disk, before doing a reinstall, and then creating the exact same partitions: the Debian installer finds the RAID signature from the previous swap partition - and automatically spins up the RAID for that partition. And then blocks any changes on that partition. Started using wipefs after finding this out this particular fact.
grub script
The script for the grub hook is also available in this repository here: uefi-copy-with-grub.