Install Debian Bookworm on a Software RAID and EFI

Posted by ads on Friday, 2024-07-19
Posted in [Hardware][Linux][Operating-Systems][Software]

Moving my systems away from Ubuntu, I also have to reinstall a server at home. This system has two equal disks for a RAID system, and my plan is to install Debian on it. Turns out, setting up a Software RAID on Debian and booting from EFI is not a straightforward approach. This blog posting describes the steps.

The “hardware”

For my use cases at home I’m using a server I bought from Thomas Krenn, it has a Intel(R) Xeon(R) E-2224 CPU @ 3.40GHz CPU and 32 GB RAM. It also has two INTEL SSDSC2KB48 SSD disks with 512 GB each. The disks are wiped to remove the previously installed Ubuntu:

1
for i in a b; do wipefs -a /dev/sd$i ; dd if=/dev/zero of=/dev/sd$i count=100000 ; done

This ensures that the Debian installer doesn’t have to deal with existing configurations. I want to again install a Software RAID on these two disks, like I previously had for Ubuntu.

First approach

The Debian disk partition tool only supports manually setting up a RAID. And the “guided” tool does not recognize when a disk/partition on a Software RAID is selected. It happily creates all the necessary partitions, including the UEFI (Unified Extensible Firmware Interface, EFI) partition, on the RAID device.

Guided installer did happily install ESP, LVM and partitions on the one RAIDed partition
Guided installer did happily install ESP, LVM and partitions on the one RAIDed partition

Of course this fails in a spectacular fashion!

Install the GRUB boot loader: Unable to install GRUB in dummy - Executing ‘grub-install dummy’ failed. This is a fatal error.
Install the GRUB boot loader: Unable to install GRUB in dummy - Executing 'grub-install dummy' failed. This is a fatal error.

Install the GRUB boot loader: Installation step failed - An installation step failed. You can try to run the failing item again from the menu, or skip it and choose something else. The failing step is: install the GRUB boot loader
Install the GRUB boot loader: Installation step failed - An installation step failed. You can try to run the failing item again from the menu, or skip it and choose something else. The failing step is: install the GRUB boot loader

Figuring out the correct RAID setup

What I need to do on the first disk is:

  • Manually create the boot block partition, mark it as biosgrub (size: 1 MB)
  • Manually create the EFI partition, mark it as EFI (size: 538 MB, same as the guided installer created)
  • Manually create the /boot partition, mark it as raid (size: 500 MB)
  • Manually create the / (root) partition, mark it as raid (remaining size minus 1.1 GB for swap in my case)
  • Manually create the swap partition, mark it as raid (remaining size)

Looks like this:

A screenshot showing the Debian Bookworm installer, Disk Partitioner, with partitions setup on one disk. The other disk is empy.

Repeat the steps for the second disk.

A screenshot showing the Debian Bookworm installer, Disk Partitioner, with partitions setup on both disks

This results in two equally configured disks. After the partitions are created, configure three (in my case) RAID devices:

  • md0: /dev/sda3 + /dev/sdb3
  • md1: /dev/sda4 + /dev/sdb4
  • md2: /dev/sda5 + /dev/sdb5

A screenshot showing the Debian Bookworm installer, Disk Partitioner, showing the available devices to create a RAID array

The RAID devices after creating the devices:

A screenshot showing the Debian Bookworm installer, Disk Partitioner, showing all RAID devices and parts of the screenshot shows partitions on the first disk

Click on each RAID device, and select what this device is used for:

  • md0 will be /boot with the 500 MB partitions
  • md1 will be / with the large 478 GB partitions
  • md2 will be swap with the 1.1 GB partitions

A screenshot showing the Debian Bookworm installer, Disk Partitioner, showing all RAID devices and parts of the screenshot shows partitions on the first disk, the RAID devices are properly configured now

From here, continue the installation. It works from here on.

After the installation

Once the installation is completed, and the system has booted into the installed system, a few additional steps are necessary.

First keep an eye on the RAID sync process:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md2 : active (auto-read-only) raid1 sda5[1] sdb5[0]
      1037312 blocks super 1.2 [2/2] [UU]
        resync=PENDING

md1 : active raid1 sdb4[0] sda4[1]
      466664448 blocks super 1.2 [2/2] [UU]
      [>....................]  resync =  1.6% (7880448/466664448) finish=37.0min speed=206184K/sec
      bitmap: 4/4 pages [16KB], 65536KB chunk

md0 : active raid1 sdb3[0] sda3[1]
      487424 blocks super 1.2 [2/2] [UU]
        resync=DELAYED

unused devices: <none>

It will take a while, but eventually all RAID devices must be synced.

In my case, the swap partition is resync=PENDING, and in auto-read-only mode. This is initially delayed until the partition receives the first writes. Can be forced with:

1
mdadm --readwrite /dev/md2
1
2
3
4
5
6
cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]

md2 : active raid1 sda5[0] sdb5[1]
      1037312 blocks super 1.2 [2/2] [UU]
      [===>.................]  resync = 19.4% (201984/1037312) finish=0.0min speed=201984K/sec

Copy the EFI files to the second disk

Currently the EFI files are only installed on the first disk (/dev/sda2 or /dev/sdb2, depending what the installer choose). In order to be able to boot from the second disk, the files need to be copied from /dev/sda2 to /dev/sdb2 or vice versa. The best place to handle this is a grub hook, which is executed whenever grub updates.

Place the following file in /etc/grub.d/90_copy_to_boot_efi2, and replace $raid_disk_first and $raid_disk_second with the device names for your primary and secondary disk.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
#!/bin/bash

set -e # abort on error
set -u # abort on variable not set
#set -x # trace execution

# one or both devices can already be mounted, potentially with a label and not with the device name

device_first="/dev/{{ raid_disk_first }}"
device_second="/dev/{{ raid_disk_second }}"

# the findmnt will fail when the device is not mounted, that is expected
set +e
mountpoint_first=`/usr/bin/findmnt --first-only --noheadings --output=TARGET ${device_first}`
mountpoint_second=`/usr/bin/findmnt --first-only --noheadings --output=TARGET ${device_second}`
set -e


if [ -z "${mountpoint_first}" ];
then
    # not mounted, create the mount point
    path_first="/efi-first"
    /usr/bin/mkdir "${path_first}"
    echo "No existing mount point found for ${device_first}"
    echo "  Mounting on ${path_first}"
    # and mount the decive
    /usr/bin/mount "${device_first}" "${path_first}"
else
    # already mounted, use the existing mount point
    path_first="${mountpoint_first}"
    echo "Found existing mount point ${mountpoint_first} for device ${device_first}"
fi

if [ -z "${mountpoint_second}" ];
then
    # not mounted, create the mount point
    path_second="/efi-second"
    /usr/bin/mkdir "${path_second}"
    echo "No existing mount point found for ${device_second}"
    echo "  Mounting on ${path_second}"
    # and mount the decive
    /usr/bin/mount "${device_second}" "${path_second}"
else
    # already mounted, use the existing mount point
    path_second="${mountpoint_second}"
    echo "Found existing mount point ${mountpoint_second} for device ${device_second}"
fi

if [ ! -d "${path_first}/EFI" -a -d "${path_second}/EFI" ];
then
    # there's an "EFI" directory in the second mount point,
    # but not in the first one - this is wrong, and will damage
    # the installation once rsync runs
    echo "There is no 'EFI' directory in the first (source) mount point!"
    echo "But there is an 'EFI' directory in the second (destination) mount point!"
    echo "This will not work, possibly switch the devices!"
    echo "Source: ${device_first} (mounted on: ${path_first})"
    echo "Destination: ${device_second} (mounted on: ${path_second})"
    exit 1
fi


#echo "rsync --dry-run --verbose --times --recursive --delete \"${path_first}/\" \"${path_second}/\""
#rsync --dry-run --verbose --times --recursive --delete "${path_first}/" "${path_second}/"
rsync --verbose --times --recursive --delete "${path_first}/" "${path_second}/"


if [ -z "${mountpoint_first}" ];
then
    # unmount the device
    /usr/bin/umount "${path_first}"
    # remove the directory
    /usr/bin/rmdir "${path_first}"
fi

if [ -z "${mountpoint_second}" ];
then
    # unmount the device
    /usr/bin/umount "${path_second}"
    # remove the directory
    /usr/bin/rmdir "${path_second}"
fi

exit 0

The {{ }} are for Ansible, I’m deploying this script using a Playbook:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
- hosts: all
  become: yes
  force_handlers: True
  vars:
    raid_disk_first: "sdb2"
    raid_disk_second: "sda2"
  tasks:

    - name: rsync packages
      ansible.builtin.apt:
        name:
          - rsync
        state: present

    - name: Upload copy_to_boot_efi2.sh
      ansible.builtin.template:
        src: "{{ playbook_dir }}/files/copy_to_boot_efi2.sh"
        dest: "/etc/grub.d/90_copy_to_boot_efi2"
        owner: "root"
        group: "root"
        mode: "0700"

And finally install the boot loader on both disks, and update grub:

1
2
3
grub-install /dev/sda
grub-install /dev/sdb
update-grub

Every time something is changed on the original EFI partition, the script will mount the other partition and sync the content.

Lessons learned

Don’t use the guided installer. It happily - and wrongly - accepts a partition on a RAID, and creates additional partitions and even an LVM on that one RAID partition. All partitions must be created manually.

When just dding the beginning of the disk, before doing a reinstall, and then creating the exact same partitions: the Debian installer finds the RAID signature from the previous swap partition - and automatically spins up the RAID for that partition. And then blocks any changes on that partition. Started using wipefs after finding this out this particular fact.

grub script

The script for the grub hook is also available in this repository here: uefi-copy-with-grub.



Comments

With an account on the Fediverse or Mastodon, you can respond to this post. Known replies are displayed below:



Note: This will load data from mastodon.social.

Learn how this is implemented here.


Share: