no way to create images with a GPT alignment of 1

[Oliver Grawert]

since it is very wasteful to create binary blob partitions for a fastboot/android based boot with a big alignment the former (ubuntu-device-flash based) dragonboard images used a 1-sector alignment with sgdisk (this is the "-a 1" option). that way there is a lot less space wasted on the SD card..

[Steve Langasek (vorlon)]

The current behavior uses an alignment of 1MiB because, as explained in the sgdisk manpage description of this option:

This alignment value is necessary to obtain optimum performance with Western Digital Advanced Format and similar drives with larger physical than logical sector sizes, with some types of RAID arrays, and with SSD devices.

Is it possible that any of these performance issues might apply to the dragonboard use case as well? Is it possible that the sizes of any of these files might have to change in a future release of the gadget snap?

And for the record, the total wasted space, across 7 partitions, by using 1MiB-aligned partitions currently instead of sector-aligned partitions, is about 3.5MiB. The current gadget snap wastes another 1MiB all on its own, by allocating a 2MiB partition for sdappsboot.mbn when this file fits in 1MiB. So I don't think this is "very wasteful" currently

That said, there's no fundamental reason for us not to allow differently-aligned partitions, for gadgets that request this; this should be under the full control of the gadget author, provided there is any sensible reason to ever deviate from this default. But it's more than just the sgdisk arguments that have to be changed, if you look through the ubuntu-image code you'll see several places that a 1MiB alignment has been assumed initially during development (always with the intent of changing this later).

[Oliver Grawert (ogra)]

well, trying to create a beaglebone gadget the sum of requirments hits hard here ...

i can not have a structure definition without partition when writing a raw img file.
the bbb requires the spl (MLO) to sit at an offset with 128k and the u-boot binary at an offset of 384k. since i am required to use a partition around these and teh minimal partition size is 512k i can not really create either of them.

device-tree: am335x-boneblack
device-tree-origin: kernel
volumes:
  beagle:
    schema: mbr
    bootloader: u-boot
    structure:
      - type: 0
        size: 131072
        content:
          - image: MLO
      - type: 0
        size: 393216
        content:
          - image: u-boot.img
      - type: 0C
        filesystem: vfat
        filesystem-label: system-boot
        size: 128M

[Oliver Grawert (ogra)]

(well, the above is a bad example sine it is mbr based ... but switching it to GPT doesnt really fix anything indeed and has the same problem)

[Liming Wang (wangliming)]

I also has the same requirement to write raw files to image.
My original snap.yaml is as below:

name: artik10-snappy-gadget
summary: Artik10 board support package
description: Support files for booting the artik10 board
version: 0.1
type: gadget
architectures: [armhf]

immutable-config:
    - ubuntu-core/autopilot*

gadget:
  branding:
    name: Artik10
    subname: Artik10

  hardware:
    platform: exynos5422
    architecture: armhf
    partition-layout: minimal
    bootloader: u-boot
    boot-assets:
      files:
          - path: uboot.env
      raw-files:
          - path: bl1.bin
            offset: 512
          - path: bl2.bin
            offset: 15872
          - path: u-boot.bin
            offset: 32256
          - path: tzsw.bin
            offset: 1080832
          - path: params_sdboot.bin
            offset: 1342976

I want to convert it to gadget.yaml, can current ubuntu-image create such image?

[Gustavo Niemeyer (niemeyer)]

This looks like another example where ubuntu-image is trying to outsmart the gadget.yaml definition. If there is benefit for some platforms to use 1MB-aligned allocation, this knowledge should be in the gadget rather than in ubuntu-image code, precisely for this sort of reason. Different platforms will have different requirements, and the generated images should not change based on code changes performed in ubuntu-image itself.

[Oliver Grawert (ogra)]

well, it simply uses the default from sgdisk in the backend, sgdisk wants a specific option (-a 1) which ubuntu-image just misses ... i dont think ubuntu-image tries to be smart on purpose here :)

i'd like to note again though that the requirement for having a partition around each blob at all means that the actual boot partition moves to a higher number, bootloaders that need to know this number (like uboot) will need to be patched to look in a different place with this requirement. perhaps we should just leave sdgisk alone and re-think the requirement for having a partition around everything. (or even fix both)

[Steve Langasek (vorlon)]

On Fri, Oct 14, 2016 at 10:42:32AM -0000, Oliver Grawert wrote:
> i'd like to note again though that the requirement for having a
> partition around each blob at all means that the actual boot partition
> moves to a higher number, bootloaders that need to know this number
> (like uboot) will need to be patched to look in a different place with
> this requirement.

No, because there's no requirement that partition numbers be in disk order.
Partition numbers will be assigned in the order the partitions appear in the
gadget.yaml, but you can use 'offset' to place higher-numbered partitions
earlier on the disk. This should give you full control of the partition
numbers for the uboot interface.

[Oliver Grawert (ogra)]

well, except that a board like the beaglebone would not boot if the MLO blob does not sit at the start of the disk and the u-boot binary does not sit byte-correct in the right position the MLO looks for it. the partition table will thus still have two partitions before the "system-boot" vfat.

even if i call the MLO and u-boot partitons 5 and 7 their position will still be 1 and 2 in the table that uboot sees and the vfat will become the third partition (while a default uboot would look on partition 1 for its env).

or do i misunderstand something ?

[Barry Warsaw (barry)]

Actually, we do sort structure parts (i.e. partitions) by offset currently.

[Barry Warsaw (barry)]

I'm marking this bug as Fix Committed for 0.15 because I think the alignment issues have been largely resolved as part of LP: #1630709

In particular, we no longer use sgdisk, but instead libparted for all partition creation. (sfdisk still has to be used to set the partition type, and it's used to gather diagnostics, but that's it.)

The only limitation that still exists is that partition offsets must be a multiple of the sector size, which defaults to 512 bytes of course. This is a limitation of libparted, so it's infeasible to change. However, libparted tells us what the device's sector size is so for a well defined gadget, this isn't hardcoded in ubuntu-image any more.

Feel free to reopen this bug if necessary, or create a new one if you find that 0.15 still imposes unnecessary or hardcoded restrictions.