Invalid backup GPT header on tegra igx image

[Rémy MARTIN]

Affects Tegra images daily builds and release image:
https://cdimage.ubuntu.com/nvidia-tegra/ubuntu-server/jammy/daily-preinstalled/current/
https://cdimage.ubuntu.com/releases/jammy/release/nvidia-tegra/

Image definition: https://git.launchpad.net/ubuntu-images/tree/ubuntu-server-tegra-igx.yaml?h=jammy
Gadget yaml: https://git.launchpad.net/~canonical-foundations/snap-pc/+git/github-mirror-amd64/tree/gadget-arm64.yaml?h=classic

Building the image:
ubuntu@ubuntu:~/ubuntu-images$ sudo /snap/bin/ubuntu-image classic -w work -O output ubuntu-server-tegra-igx.yaml
WARNING: rootfs.sources-list-deb822 was not set. Please explicitly set the format desired for sources list in your image definition.
WARNING: rootfs.sources-list-deb822 is set to false. The deprecated format will be used to manage sources list. Please if possible adopt the new format.
[0] build_gadget_tree
[1] prepare_gadget_tree
[2] load_gadget_yaml
WARNING: volumes:pc:structure:0:filesystem_label used for defining partition roles; use role instead
[3] verify_artifact_names
[4] germinate
[5] create_chroot
[6] install_packages
[7] prepare_image
[8] preseed_image
[9] clean_rootfs
[10] customize_sources_list
[11] customize_cloud_init
[12] set_default_locale
[13] populate_rootfs_contents
[14] calculate_rootfs_size
[15] populate_bootfs_contents
[16] populate_prepare_partitions
[17] make_disk
[18] update_bootloader
[19] generate_package_manifest
Build successful

Checking the partition table:
ubuntu@ubuntu:~/ubuntu-images$ gdisk -l output/ubuntu-22.04-preinstalled-server-arm64+tegra-igx.img
GPT fdisk (gdisk) version 1.0.8

Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!
Main header: OK
Backup header: ERROR
Main partition table: OK
Backup partition table: ERROR

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************

Warning! Secondary partition table overlaps the last partition by
9585020 blocks!
You will need to delete this partition or resize it in another utility.
Disk output/ubuntu-22.04-preinstalled-server-arm64+tegra-igx.img: 9724252 sectors, 4.6 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): F29EC7AD-406F-4DD2-89BF-94F539E6BFAC
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 139231
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number Start (sector) End (sector) Size Code Name
   1 2048 104447 50.0 MiB EF00 EFI System
   2 104448 9724251 4.6 GiB 8300 writable

[Paul Mars (upils)]

Hey Rémy,

We are aware of issues regarding the partition layout, including the GPT backup header (see LP: #2045006, and https://github.com/canonical/ubuntu-image/pull/124).

Thank you for the detailed use case, it will surely help, at least to write a test case. I currently plan on investigating this right after Madrid.

[Rémy MARTIN (rmartin013)]

Hey Paul,

Thanks for this feedback. It seems this error is related to the gadget snap. I made a try building for instance https://git.launchpad.net/ubuntu-images/tree/ubuntu-server-pc-arm64.yaml?h=noble which uses the very same gadget snap, and the result slightly differs

ubuntu@ubuntu:~/ubuntu-images/output$ parted ../output-noble/ubuntu-24.04-preinstalled-server-arm64.img print
WARNING: You are not superuser. Watch out for permissions.
Warning: Unable to open /home/ubuntu/ubuntu-images/output-noble/ubuntu-24.04-preinstalled-server-arm64.img read-write (Permission denied).
/home/ubuntu/ubuntu-images/output-noble/ubuntu-24.04-preinstalled-server-arm64.img has been opened read-only.
Error: The backup GPT table is not at the end of the disk, as it should be. Fix, by moving the backup to the end (and removing the old backup)?
Fix/Ignore? ^C
Warning: Not all of the space available to /home/ubuntu/ubuntu-images/output-noble/ubuntu-24.04-preinstalled-server-arm64.img appears to be used, you can fix the GPT to use all of the space (an extra 5637383
blocks) or continue with the current setting?
Fix/Ignore? ^C
Model: (file)
Disk /home/ubuntu/ubuntu-images/output-noble/ubuntu-24.04-preinstalled-server-arm64.img: 2958MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
 1 1049kB 53.5MB 52.4MB fat32 EFI System boot, esp
 2 53.5MB 2958MB 2904MB ext4 writable

But we don't see that on the arm64 generic server image. Do you know how is it build (image definition, gadget, u-i arguments)?

[Paul Mars (upils)]

Here https://ubuntu-archive-team.ubuntu.com/cd-build-logs/ubuntu-server/noble/daily-preinstalled-20240422.1.log you can get the link to the ubuntu-server-arm64-generic log https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/noble/cpc/+build/607831 . And you can see it was not built using ubuntu-image.

[Paul Mars (upils)]

You may also want to tweak the image size using the ``--image-size` option. See the doc https://canonical-subiquity.readthedocs-hosted.com/en/latest/reference/ubuntu-image.html

[Rémy MARTIN (rmartin013)]

Interesting... There are so many different ways it is hard to follow and figure out what is the right way. Anyway, you are right, the point here is about the gadget yaml. I made a try yesterday defining a structure for system-data, had to define a size as well, and doing so "works around" the issue. I proposed a PR (https://github.com/snapcore/pc-gadget/pull/118) that should fix our builds once merged. Alternatively, we can use that image-size option if you prefer not to change the gadget file.

But this would definitely deserve a proper fix, as the rootfs could slightly grow over time...

Can you please tell me if we can quickly fix the issue merging that PR?

[Paul Mars (upils)]

Image building is indeed confusing.

I believe this gadget is used for other images, and I fear having a 5Gb partition might break some use cases (especially until we support building sparse images).

I do not have enough permissions to merge on this GH project sadly. I pinged waveform and sil2100 so they can take a look.

Even though I understand we try to avoid dealing with too many branches on this project, we might also merge your changes in a branch that you could reference in the image definition. Currently the "classic" branch is used, but you could use a "classic-tegra" one. This might be at least a temporary solution to unblock you until we confirm or not we can add this structure in the gadget.yaml.

I also agree a proper fix would be best. My current understanding is that we need to make sure each structure has enough space to avoid mangling the next one.

[Rémy MARTIN (rmartin013)]

Thanks. I am fine with using a classic-tegra branch. I will recreate a PR and also create a MR for the image definition file, to merge only when the GH change will land on the mirrored repo.

[Rémy MARTIN (rmartin013)]

Actually, it turns out I cannot create a new branch. I guess only waveform and sil2100 can do so, they probably also can simply edit my PR and switch the destination branch.

[Rémy MARTIN (rmartin013)]

Created the 2nd MR: https://code.launchpad.net/~rmartin013/ubuntu-images/+git/ubuntu-images/+merge/465506