[Ubuntu 22.04] Allow ESP installation on md-disks in subiquity

[Sujith Pandel]

subiquity installer currently does not allow to select md-disk as Installation destination where /, /home and /boot/efi could be created and installed.
Request is to enable this feature so that we can install and boot from md-disk, which our BIOS creates.

[Michael Reed (mreed8855)]

This bug is an updated version of an older issue. https://bugs.launchpad.net/bugs/1466150

[Dan Bungert (dbungert)]

So I see the linked bug about raid.

Can you clarify for me what md-disk refers to in this context? Search results are pointing me to MiniDisk and Markdown and those are obviously not correct.

[Sujith Pandel (sujithpandel)]

Our BIOS uses mdadm and creates md-raid disks (RAID-1) with metadata 1.2 through Dell SW-RAID solution like S140/S150 etc inside BIOS.
We support install and boot from these md-raid disks.

https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm
https://dl.dell.com/topicspdf/s150-users-guide_en-us.pdf

[Sujith Pandel (sujithpandel)]

Steps:
1. Setup a DellEMC 14G PE server with 2 Onboard SATA disks.
2. Setup a md-raid (RAID1) on these SATA disks through BIOS S140 controller in UEFI mode.
3. Start installing Ubuntu 22.04
4. See that user cannot select md-raid disk as boot disk. Installation on md-raid disk cannot proceed.

[Sujith Pandel (sujithpandel)]

Description of the feature request:

The DellEMC PowerEdge RAID Controller(PERC) S130/S140 is RAID solution for the DellEMC PowerEdge systems configured through Chipset SATA controller. It does not use any dedicated Hardware for RAID operations. It is set up/configured by the system BIOS. Using the BIOS/UEFI Configuration utility user can configure RAID on onboard SATA/NVMe Drives and install OS and boot from it.

To make use of S130/S140 solution, the user need to change the SATA mode to RAID in BIOS/UEFI Configuration.
During S130/S140 configuration, the VDs (Virtual Disk) are created by system BIOS by writing metadata on the selected onboard SATA or NVMe drives. The metadata format used is MD version 1.2. The resulting VD is a Linux MDRAID VD.

This is the feature that we are requesting - to get install and boot support for Ubuntu 22.04 and future releases of Ubuntu 20.04

[Julian Andres Klode (juliank)]

This is very unfortunate. So the UEFI system is able to read mdraid 1.2 too, and can access ESPs inside of it?

Most systems do not implement linux's mdraid in their firmware, and hence putting the ESP inside the RAID would lead to unbootable systems; which is why we designed the solution where you can place an ESP on each disk outside the raid and then raid the partitions instead.

There's some caveats in that systems accidentally can read ESPs in an mdraid 1.0 as the metadata is at the end only, but in general exposing that would be a bad idea.

So I guess one could implement a quirk to allow you to place ESPs in there on this specific hardware, but creating such RAIDs and placing ESPs in them in general should not be possible.

[Sujith Pandel (sujithpandel)]

>So the UEFI system is able to read mdraid 1.2 too, and can access ESPs inside of it?

Yes, currently install & boot with mdraid 1.2 works for other OS like CentOS etc.

Ubuntu subiquity installer seems to be blocking it for ESP/Boot-device.
Ubuntu legacy debian installer was allowing md-raid disk to be selected as Installation destination (including ESP), however we were facing grub install error during installation as in https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1776685 and https://bugs.launchpad.net/ubuntu/+source/grub-installer/+bug/1466150

[Sujith Pandel (sujithpandel)]

I believe the code changes has to go to these files:
https://github.com/canonical/subiquity/blob/main/subiquity/common/filesystem/boot.py
and
https://github.com/canonical/subiquity/blob/main/subiquity/common/filesystem/actions.py

Currently it allows only imsm containers and hence our md-raid 1.2 disks aren't being considered as Boot drive candidate.

What kind of subiquity quirk do you propose to enable Linux md-raid 1.2 disks?

[Sujith Pandel (sujithpandel)]

Also, we feel even after subiquity allows mdraid-disk installations, we might be hitting https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1776685 here as well. So we need to work on that as well to get a complete working solution.

[Michael Hudson-Doyle (mwhudson)]

You are correct that subiquity does not allow you to put the ESP on a metadata 1.2 MD RAID device. That's because, to the best of my knowledge, almost all firmware does not know about MD RAID formats and so would be unable to boot the system.

The question thus becomes, how can the installer know that putting the ESP on a metadata 1.2 MD RAID is going to work? I presume we can identify your firmware by looking at sysfs somehow or other?

I still wouldn't be amazingly happy about doing this because it is a general goal of Ubuntu's installers that you can transfer a disk between systems and still have it boot, but that's probably not so important for the server use case.

[prabhakar pujeri (prabhakarpujeri)]

Hi Michael,

I am checking internally if we have any way to tell bootable 1.2 metadata or not.

but all Dell PowerEdge systems are boot supported with 1.2 metadata. we can filter with this for dell.

[prabhakar pujeri (prabhakarpujeri)]

Hi Michael,

we had an internal discussion with our SW RAID team and we don't have any option in metadata to tell the booting disk or data disk. The only way suggested is to filter the Dell server and all dell servers have support for mdraid boot.