[2.0b6,arm64] maas installs to USB drive if it is present instead of SSD

[Manoj Iyer]

My arm64 server has SSD, SD and USB drives. When I commission maas shows me possible targets as SSD (dev/sda) and SD (mmcblk0) but does not list the USB drive. I can then remove the SD drive from targets from the UI. But, when I deploy maas picks the USB drive as the install target instead of the SSD (dev/sda)even though sda-part1 and sda-part2 are the two storage file systems listed in MAAS UI it shows no other disks or available partitions. This causes maas to fail to boot from disk ie ssd (dev/sda) after deployment. The only workaround for now is to physically remove any USB.

[Sean Feole (sfeole)]

Appended lshw output and block-devices.out to the bug

[Andres Rodriguez (andreserl)]

That is strange. Can you attache the output of:

maas <maasuser> node get-curtin-config <system id>

[Andres Rodriguez (andreserl)]

Actually:

maas <maasuser> machine get-curtin-config <system id>

[Sean Feole (sfeole)]

attached. Starting to have my doubts on this, the storage config looks ok.

[Blake Rouse (blake-rouse)]

Looking at this everything looks correct from the MAAS side. This entry in the curtin config tells curtin what device to use, it does this by model/serial number:

storage:
  config:
  - grub_device: true
    id: sda
    model: Samsung SSD 845D
    name: sda
    ptable: gpt
    serial: S1RGNWAG301454J
    type: disk
    wipe: superblock

Are you sure that its not installing to the correct disk, but when it reboots it is booting from an old installation of Ubuntu on the thumbdrive? If you would keep the USB thumbdrive defined in MAAS, it would wipe the superblock on installation to make sure that it does not boot from that device.

If your still really sure that it is installing to the thumbdrive please enable debug mode for curtin and reinstall. Attach the installation log from the installation once it fails.

maas my-maas-session maas set-config name=curtin_verbose value=true

[Manoj Iyer (manjo)]

Removing the USB stick or (perhaps reformatting the USB stick) fixes the issue. MAAS UI provides the option to remove SD cards after commissioning, but it does not list any USB sticks as possible targets so there is no way of removing possible USB targets from MAAS. Nor does it give you an option to format select disks on commissioning/deploy so that they dont mess up the boot process. If the USB stick has a valid grub entry it seems like this confuses maas and tries to boot from it when it chainloads. Blake might be right that MAAS installs to SSD, but it definitely is confused as to what target to boot after deploy coz it sees valid grub files on the USB.

The other theory that someone suggested is that this might have to do with UEFI, I cant see that as a possibility, as far as uefi boot order is concerned, USB is listed as the last option here. So I cant see any possible reason as to how eufi would prefer to boot from usb.

[Blake Rouse (blake-rouse)]

Booting UEFI vs Legacy has an affect, which is the machine booting?

[Andrew Cloke (andrew-cloke)]

UEFI...

[Blake Rouse (blake-rouse)]

Here is the boot process for UEFI:

1. UEFI Starts
2. DHCP from MAAS
3. Loads bootx64.efi from MAAS
4. bootx64.efi loads grubx64.efi from MAAS
5. grubx64.efi loads grub/grub.cfg from MAAS
6. grub/grub.cfg loads grub/grub.cfg-{mac-address} from MAAS
7. grub/grub.cfg-{mac-address} searches for grubx64.efi on a local disk
8. chainloads the found grubx64.efi from the localdisk

#7 is where things can go wrong. The order of the search is not determined by the BIOS it is determined by grub and you cannot control that order and there is no way to always make it the same order. In this case you have a grubx64.efi on the USB thumbdrive as well. So grub finds that grubx64.efi and boots that.

The only real way to fix this would be to make sure that curtin uses the UUID that MAAS provides it for the partition that has '/boot' on it. Then modify #7 to search for the disk based on that UUID and load the grubx64.efi from that disk instead of any disk that has grubx64.efi.

I am also interested in why that thumbdrive is not showing up in MAAS as a usable device.