ubuntu-server ARM64 ISOs lack partition table with EFI System Partition

[Paweł Moll]

[Impact]
On some arm64/efi platforms, the firmware will not be able to boot from an ISO dd'd to a USB stick.

[Test Case]
My test is to attach the ISO as an "HD/USB Image" to a Cavium CRB1S w/ AMI UEFI firmware using the BMC's Virtual Media. Note that the BMC requires that the file be renamed with a ".img" suffix.

[Regression Risk]
For d-i, this is aligning the xorriso boot options with the same ones now used to build the server ISO. Regression risk has been mitigated by testing on real hardware connected as both CD and disk images.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1692876/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Paweł Moll (pawel-moll)]

> [This is an automated message.  I apologize if it reached you
> inappropriately; please just reply to this message indicating so.]

As far as I can say (even after reviewing the wiki page) there's no
relevant source package for that issue, not even the debian-installer
one, as the ISO images are built by an "external" infrastructure...

[Adam Conrad (adconrad)]

You should be able to write the ISO directly to a USB stick with dd (ie: "dd if=image.iso of=/dev/sdc bs=1M") and it should boot fine on your EFI system. I suspect usb-creator did nasty things to it.

[Adam Conrad (adconrad)]

Oh, I missed the bit where this said arm64, not amd64. Perhaps there's a bit more to investigate here.

[PRAKASH (bprakash)]

Is there any update on this issue

[Paweł Moll (pawel-moll)]

Prakash: you may want to click "This bug affects me" if it affects you ;-)

Adam: well, my bet is that the AMD64 (x86_64) ISO build scripts have been upgraded to build the "hybrid" image, while the ARM64 (AArch64) haven't been treated equally.

[Steve Langasek (vorlon)]

It looks to me like this may have been broken by revno 1945 on debian-cd, which dropped the population of $CDDIR/boot/grub/efi.img. Mathieu also mentions that hybrid boot would require the options -eltorito-alt-boot -e boot/grub/efi.img -no-emul-boot -isohybrid-gpt-basdat. Dann, is this something you can drive resolution of?

[Ard Biesheuvel (ard-biesheuvel)]

In my case, UEFI can find grub just fine after I dd the .iso straight to a USB stick. However, the installer fails to mount it under /cdrom, which seems to be caused by the fact that cdrom-detect invokes list-devices in a way that dismisses USB sticks without a partition table.

The workaround is to enter the shell and mount the device on /cdrom manually, after which the installer proceeds happily.

[Paweł Moll (pawel-moll)]

That means, I believe, that your EFI can handle ISO9660 - one of my boards can do it as well (and indeed I had the same problem with /cdrom at the installer level on it). The rest - don't like it at all.

[dann frazier (dannf)]

I have verified that, if I dd a currently xenial daily ISO to a usb stick, I cannot get a Cavium CRB1S w/ AMI 018 UEFI to boot from it. However, like Ard mentioned, this does boot fine on other UEFI-based platforms (QEMU + Tianocore, HiSilicon D05).

[Paweł Moll (pawel-moll)]

Yes, I've noticed that as well - some AArch64 EFI builds can boot from "classic" ISO9660, some can't. Now, considering the fact that amd64 (as in: x86_64) clearly suffered from similar issues and have solved it by providing the EFI FAT partition, couldn't we just get the same on arm64 (as in: AArch64)?

[Thomas Schmitt (scdbackup)]

Hi,

this bug is somewhat mis-titled, because ubuntu-17.04-server-arm64.iso
does have EFI boot equipment for optical media as El Torito boot image
for "Platform" 0xEF = UEFI.
The title would better be:
  "ubuntu-server ARM64 ISOs lack partition table with EFI System Partition"

My proposal would be to do it more or less like Debian by an appended
partition. See my comments of today at
  https://code.launchpad.net/~dannf/debian-cd/lp1692876/+merge/331671

Have a nice day :)

Thomas

[Jo Shields (directhex)]

For reference, I was affected by this issue on a Softiron Overdrive 3000 (AMI firmware, AMD SoC)

[dann frazier (dannf)]

@directhex: would you mind verifying that this resolves the issue for you as well? You can try either any (or all!) of:
 A recent daily build:
  http://cdimage.ubuntu.com/ubuntu-server/xenial/daily/current/
  http://cdimage.ubuntu.com/ubuntu-server/daily/current/

 My test build of the d-i installer:
  http://ppa.launchpad.net/dannf/test/ubuntu/dists/xenial/main/installer-arm64/current/images/hwe-netboot/mini.iso

I don't know if any of these Ubuntu releases support your platform, but they should at least boot into GRUB.

[Jo Shields (directhex)]

I'll give it a shot when I get to the office tomorrow

[Jo Shields (directhex)]

Confirmed: your test mini.iso comes up as FS1 in the UEFI shell when dd'd to a hard disk, and boots fine. Currently running through the installer. Working absolutely fine thus far (installing base system)

[Launchpad Janitor (janitor)]

This bug was fixed in the package debian-installer - 20101020ubuntu527

---------------
debian-installer (20101020ubuntu527) bionic; urgency=medium

  [ dann frazier ]
  * arm64-efi/netboot: Improvements to mini.iso generation:
    - Add an EFI System Partition to fix booting from USB stick on
      same platforms LP: #1692876.
    - Remove no-op arguments -boot-load-size and -boot-info-table.
    - Use simpler "-e" instead of "--efi-boot" since we only have 1
      el torito image.
    - Add -partition_offset 16 so that the output of commands like
      'isosize' is correct.

  [ Adam Conrad ]
  * Move master kernels to 4.13.0-25.

 -- dann frazier <email address hidden> Wed, 03 Jan 2018 11:39:33 -0700

[Łukasz Zemczak (sil2100)]

Hello Paweł, or anyone else affected,

Accepted debian-installer into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debian-installer/20101020ubuntu451.22 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dann frazier (dannf)]

Verified: I was able to boot the mini.iso on a Cavium Sabre system from a USB stick.

[Launchpad Janitor (janitor)]

This bug was fixed in the package debian-installer - 20101020ubuntu451.22

---------------
debian-installer (20101020ubuntu451.22) xenial; urgency=medium

  [ dann frazier ]
  * arm64-efi/netboot: Improvements to mini.iso generation:
    - Add an EFI System Partition to fix booting from USB stick on
      same platforms LP: #1692876.
    - Remove no-op arguments -boot-load-size and -boot-info-table.
    - Use simpler "-e" instead of "--efi-boot" since we only have 1
      el torito image.
    - Add -partition_offset 16 so that the output of commands like
      'isosize' is correct.
  * netboot/arm64: Include nic-firmware. LP: #1743638.

  [ Łukasz 'sil2100' Zemczak ]
  * Bump FLOPPY_SIZE on amd64, i386, and powerpc for kernel growth.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 21 Feb 2018 10:43:54 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for debian-installer has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.