bios/efi maas deployment failure with drives over 2TB

[James]

MAAS version: 2.2.2 (6099-g8751f91-0ubuntu1~16.04.1)
Hardware: Dell R710 6 (2TB) Drives in RAID6 using a perc6/i RAID controller.

During BIOS boot, deployment passes with no errors but upon initial reboot into OS, boot fails with:

"attempt to read or write outside of disk 'hd0'
Entering rescue mode..."

error (Installation log viewable here: https://pastebin.com/4YdiSBmF )

With EFI enabled, deployment fails with attached installation log.

curtin-config:
https://pastebin.com/yWnaEE64

[James (james-o-benson)]

Note:
Deployment passes and boots properly with maas v2.1.5 and custom partitioning:

/boot 1GB
SWAP 4GB
/ ~8TB

[Ryan Harper (raharper)]

Hi,

If your machine is booting in EFI mode, MAAS needs to know (I believe you need to recommission it if you change the setting in your bios);

The curtin storage config sent is not configure for UEFI booting, in particular MAAS would have instructed the creation of a partition for /boot/EFI mount point.

Booting grub with 2TB drives is possible with GPT partition, which is set in the configuration, however, it does not include a grub_bios partition, which is needed, something like:

      - id: bios_boot
        type: partition
        size: 1MB
        number: 15
        device: main_disk
        flag: bios_grub

I wonder if the raid array was assembled after you commissioned the system? If MAAS doesn't know the storage devices are larger than 2TB it might not send the right configuration to add the grub_bios flag.

[Ryan Harper (raharper)]

For the 2.1.5 deployment that passes, can you share the curtin config that is used?

[James (james-o-benson)]

Ryan, I did as you suggested. I enabled UEFI and re-commissioned my machine (I did not do this earlier). This did change the default storage options to:

sda-part1 536.9 MB fat32 /boot/efi
sda-part2 8.0 TB ext4 /

Installation is successful as well:
https://pastebin.com/ZFuKwGGk

However, upon boot, I am greeted with:

Booting local disk...
error: no such device: /efi/ubuntu/shimx64.efi
error: File not found.

I'll release the machine again. In regards to the bios_boot partition mentioned in comment 2. How do you specify the number option and 1MB size ( 5MB is the smallest I can choose)? Also, this is under BIOS, not EUFI correct (just trying to be clear)? Should the flag be bios_grub or grub_bios? You use both versions.

In regards to your question, the RAID is independent of the BIOS/UEFI option but I did not commission again after I changed from BIOS to UEFI earlier, hence the different result.

[Ryan Harper (raharper)]

On Wed, Sep 13, 2017 at 11:09 PM, James <email address hidden> wrote:

> Ryan, I did as you suggested. I enabled UEFI and re-commissioned my
> machine (I did not do this earlier). This did change the default storage
> options to:
>
> sda-part1 536.9 MB fat32 /boot/efi
> sda-part2 8.0 TB ext4 /
>
> Installation is successful as well:
> https://pastebin.com/ZFuKwGGk
>
> However, upon boot, I am greeted with:
>
> Booting local disk...
> error: no such device: /efi/ubuntu/shimx64.efi
> error: File not found.
>

This is an existing grub issue; let me find the bug.

https://bugs.launchpad.net/maas/+bug/1687729

> I'll release the machine again. In regards to the bios_boot partition
> mentioned in comment 2. How do you specify the number option and 1MB
> size ( 5MB is the smallest I can choose)? Also, this is under BIOS, not
> EUFI correct (just trying to be clear)? Should the flag be bios_grub or
> grub_bios? You use both versions.
>

MAAS knows how to do this; you shouldn't need to do anything.

For BIOS booting with GPT, grub needs a place on disk to stuff some of it's
loaders, on MBR/MSDOS there was some "empty" space that grub used, with
GPT, that doesn't exist any more, so grub needs a special partition labeled
for grubs use. 5MB is fine, 1MB is more than enough space. The flag to use
is 'bios_grub'. Sorry for the confusion.

And, you shouldn't have to input any of that. MAAS should know that your
machine
is in BIOS vs. UEFI after commissioning, and if the storage devices
detected are
> 2TB it should set the partition table to GPT and include a bios partition
and set the
flag accordingly.

If you're seeing that commissioning in BIOS mode and with disks of size >
2TB do not
generate a storage configuration that's bootable, then there's a MAAS bug
to get fixed.

>
> In regards to your question, the RAID is independent of the BIOS/UEFI
> option but I did not commission again after I changed from BIOS to UEFI
> earlier, hence the different result.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1717031
>
> Title:
> bios/efi maas deployment failure with drives over 2TB
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/curtin/+bug/1717031/+subscriptions
>

[Andres Rodriguez (andreserl)]

James,

can you provide the whole curtin config after you commissioned the machineD?

maas <user> machines get-curtin-config <system-id>

[James (james-o-benson)]

Here is my process:
1. set UEFI in the BIOS
2. commission the machine

3. Kept everything default in maas
  536.9 MB fat32 /boot/efi - NO MOUNT OPTIONS.
   10.0 TB ext4 /
3.1 Curtin results:
http://paste.ubuntu.com/25982879/
3.2 Deploy failed with message:

   Booting local disk...
   error: no such device: /efi/ubuntu/shimx64.efi.
   error: File not found.

   Press any key to continue...

GNU GRUB version 2.02~beta2-36ubuntu3.14

4. custom partitioning
   1 GB fat32 /boot/efi - 'boot' option.
   10 GB SWAP
   10 TB ext4 /
4.1 Curtin results:
http://paste.ubuntu.com/25983227/
4.2 Deployment failure.
NOTE: Can not make an unpartitioned 1MB partition with label: bios_grub

5. BIOS not UEFI works.

[Ryan Harper (raharper)]

On Fri, Nov 17, 2017 at 3:31 PM, James <email address hidden> wrote:

> Here is my process:
> 1. set UEFI in the BIOS
> 2. commission the machine
>
> 3. Kept everything default in maas
> 536.9 MB fat32 /boot/efi - NO MOUNT OPTIONS.
> 10.0 TB ext4 /
> 3.1 Curtin results:
> http://paste.ubuntu.com/25982879/
> 3.2 Deploy failed with message:
>
> Booting local disk...
> error: no such device: /efi/ubuntu/shimx64.efi.
> error: File not found.
>
> Press any key to continue...
>
> GNU GRUB version 2.02~beta2-36ubuntu3.14
>

Related bugs I think:

https://bugs.launchpad.net/maas/+bug/1647827
https://bugs.launchpad.net/maas/+bug/1711203

Both of which cover a PXE boot + UEFI + shim sort of error
which looks like your seeing.

>
>
> 4. custom partitioning
> 1 GB fat32 /boot/efi - 'boot' option.
> 10 GB SWAP
> 10 TB ext4 /
> 4.1 Curtin results:
> http://paste.ubuntu.com/25983227/
> 4.2 Deployment failure.
> NOTE: Can not make an unpartitioned 1MB partition with label: bios_grub
>

If your node is still considered UEFI to MAAS, I don't think it will let
you make the bios_grub partition.

If you commision in bios mode, then use GPT as the partition table type; it
should let you create
something like this:

      - id: bios_boot
        type: partition
        size: 1MB
        number: 15
        device: main_disk
        flag: bios_grub

The partition number doesn't matter, just the flag and location.
And I think we learned, suggested that the boot partition be placed
close to the start of the disk so as the offset to the boot partition is not
beyond 2TB.

I think that means it would need to be defined before adding the root
partition
which would exceed 2TB while in BIOS mode.

> 5. BIOS not UEFI works.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1717031
>
> Title:
> bios/efi maas deployment failure with drives over 2TB
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/curtin/+bug/1717031/+subscriptions
>

[Launchpad Janitor (janitor)]

[Expired for curtin because there has been no activity for 60 days.]

[Andres Rodriguez (andreserl)]

Hi!

**This is an automated message**

We believe this is may no longer be an issue in the latest MAAS release. Due to the original date of the bug report, we are currently marking it as Invalid. If you believe this bug report still valid against the latest release of MAAS, or if you are still interested in this, please re-open this bug report.

Thanks