arm64/xgene-uboot lacks u-boot-tools

[Andres Rodriguez]

Just filing this bug here as I didn't know where else to file it. But long story short, the maas ephemeral image for arm64/xgene-uboot lacks u-boot-tools. This needs to be available in the image.

'u-boot-tools'

[Jason Hobbs (jason-hobbs)]

u-boot-tools is the only package we need added.

[Scott Moser (smoser)]

It seems to me that u-boot-tools is happily installed in the images. at least in the latest daily.
See the attached file, whose output shows:

-rwxr-xr-x 1 root root 56316 Nov 17 2013 ./usr/bin/mkimage
-rwxr-xr-x 1 root root 18308 Nov 17 2013 ./usr/bin/fw_printenv
-rw-r--r-- 1 root root 1568 Nov 17 2013 ./usr/share/doc/u-boot-tools/changelog.Debian.gz
-rw-r--r-- 1 root root 2005 Nov 17 2013 ./usr/share/man/man1/mkimage.1.gz
lrwxrwxrwx 1 root root 11 Nov 17 2013 ./usr/bin/fw_setenv -> fw_printenv

[Andres Rodriguez (andreserl)]

254976K ........ ........ ........ ........ ........ ........ 91% 34.5M 42s
258048K ........ ........ ........ ........ ........ ........ 92% 57.4M 36s
261120K ........ ........ ........ ........ ........ ........ 93% 57.5M 31s
264192K ........ ........ ........ ........ ........ ........ 94% 55.8M 25s
267264K ........ ........ ........ ........ ........ ........ 96% 71.2M 19s
270336K ........ ........ ........ ........ ........ ........ 97% 86.2M 14s
273408K ........ ........ ........ ........ ........ ........ 98% 82.0M 8s
276480K ........ ........ ........ ........ ........ ........ 99% 35.5M 3s
279552K ........ ........ ........ ..... 100% 28.5M=7m48s

2014-07-09 18:49:19 (601 KB/s) - written to stdout [288204424/288204424]

Leaving 'diversion of /etc/init/ureadahead.conf to /etc/init/ureadahead.conf.disabled by cloud-init'
update-initramfs: Generating /boot/initrd.img-3.13.0-29-generic
df: Warning: cannot read table of mounted file systems: No such file or directory
flash-kernel: installing version 3.13.0-29-generic
Generating kernel u-boot image... /usr/sbin/flash-kernel: 233: /usr/sbin/flash-kernel: mkimage: not found
run-parts: /etc/initramfs/post-update.d//flash-kernel exited with return code 127
Unexpected error while running command.
Command: ['chroot', '/tmp/tmpYIcUGG/target', 'update-initramfs', '-u']
Exit code: 1
Reason: -
Stdout: ''
Stderr: ''
Unexpected error while running command.
Command: ['curtin', 'curthooks']
Exit code: 3
Reason: -
Stdout: ''
Stderr: ''

[Andres Rodriguez (andreserl)]

Scott, mkimage is available in u-boot-tools, which is not present in the image it seems. Or is this supposed to be present in the target?

[Scott Moser (smoser)]

The failure you're showing above is from 'chroot $TARGET update-initramfs -u'.

Curtin does this in curthooks because curtin/commands/curthooks.py:
    # As a rule, ARMv7 systems don't use grub. This may change some
    # day, but for now, assume no. They do require the initramfs
    # to be updated, and this also triggers boot loader setup via
    # flash-kernel.
    machine = platform.machine()
    if machine.startswith('armv7'):
        update_initramfs(target)
    else:
        setup_grub(cfg, target)

There are a few things I'm not sure of:
 a.) why 'mkimage' isnt available inside your target.
   If the target was created from the ephemeral image (which is the normal process) then it would be, as we've demonstrated above that the ephemeral image has 'mkimage'.
 b.) if curtin *should* call 'update_initramfs' in the target on this system.
 c.) if mkimage is a requirement of update-initramfs, then why is it not a dependency of initramfs-tools.
      I'm admittedly not fully aware of the relationship of initramfs-tools and mkimage, but personally the above seems like a bug in update-initramfs or its packaging.

[Andres Rodriguez (andreserl)]

The output in comment #2 is for armhf images and not for arm64. The output for arm64 images is as follows:

roaksoax@pursue:~$ ./image.sh
--2014-07-14 09:28:20-- http://maas.ubuntu.com/images/ephemeral-v2/daily/trusty/arm64/20140618.1/root-image.gz
Resolving maas.ubuntu.com (maas.ubuntu.com)... 91.189.90.19, 91.189.89.122
Connecting to maas.ubuntu.com (maas.ubuntu.com)|91.189.90.19|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 305494648 (291M) [application/x-gzip]
Saving to: ‘/tmp/root-image.gz.tmp’

100%[=====================================================================================================================================================================================================>] 305,494,648 855KB/s in 5m 39s

2014-07-14 09:33:59 (881 KB/s) - ‘/tmp/root-image.gz.tmp’ saved [305494648/305494648]

ls: cannot access ./usr/bin/mkimage: No such file or directory
ls: cannot access ./usr/bin/fw_printenv: No such file or directory
ls: cannot access ./usr/share/doc/u-boot-tools/changelog.Debian.gz: No such file or directory
ls: cannot access ./usr/share/man/man1/mkimage.1.gz: No such file or directory
ls: cannot access ./usr/bin/fw_setenv: No such file or directory

[Scott Moser (smoser)]

I've made this bug apply to 'flash-kernel'. My understanding of the issue is
 * flash-kernel is called by update-initramfs.
 * flash-kernel has some logic that decides to invoke mkimage sometimes

it seems to me that flash-kernel package should at very least 'Recommend' u-boot-tools at least on armhf or arm64 systems.
At worst case, such a recommend would mean ubuntu systems (where apt recommends by default) would get u-boot-tools on systems where it would go un-used. u-boot-tools is installed size of 172k. I think that is reasonable cost to hide such bugs from future users.

[dann frazier (dannf)]

The relationship between these is as follows.

flash-kernel requires u-boot-tools on certain platforms. This is detected at d-i installtime by flash-kernel-installer like so:

dannf@mustang:~$ export FK_DIR="/usr/share/flash-kernel"
dannf@mustang:~$ . $FK_DIR/functions
dannf@mustang:~$ machine="$(get_cpuinfo_hardware)"
dannf@mustang:~$ echo $machine
APM X-Gene Mustang board
dannf@mustang:~$ get_machine_field "$machine" "Required-Packages"
u-boot-tools
dannf@mustang:~$

Presumably this is because arm platforms have historically been embedded, so it made sense to make this a dynamic decision vs. a hard package dependency to avoid installing unnecessary packages where possible. That's just a guess though.

flash-kernel installs an initramfs-hook that causes flash-kernel to be executed when the initramfs is regenerated. If the platform requires u-boot-tools, it expected u-boot-tools to be there. Now, it could be argued that flash-kernel's initramfs-tools hook shouldn't exit with an error in this situation (due to a missing but undeclared dependency), and perhaps instead print a large warning to the console - that's an easy enough fix. But I don't know of a good way to make a non-d-i-installer know what packages d-i would install on a target platform w/o some higher level redesign.

[Oliver Grawert (ogra)]

(teh actual duplicate would be bug 1160488 (which has an explanation from lool why a dependency is not appropriate ... but LP does not let me duplicate to duped bugs)

[Scott Moser (smoser)]

i dont see any sort of explanation as to why a dependency is not appropriate. I see
a.) stgraber agreeing that there should be a dependency
   https://bugs.launchpad.net/ubuntu/+source/flash-kernel/+bug/1160360/comments/5
b.) loic suggesting that some other piece of software should be used to decide which packages to install
   https://bugs.launchpad.net/ubuntu/+source/flash-kernel/+bug/1160488/comments/1

why is it not possible to do the right thing?

Why couldn't whatever logic is used to decide "should I install flash-kernel" instead be used at runtime?

[Loïc Minier (lool)]

Basically, currently the Debian/Ubuntu approach is that different platforms need different boot setups; e.g. legacy BIOS vs. EFI, ARM vs x86, this or that specific platform needs this or that specific boot package.

The way the right packages for boot get pulled on live images and on debian-installer images is by running the logic in udeb packages (flash-kernel-installer.postinst).

Either you dont need a boot update solution, and then you should not installed it as outlined in:
https://bugs.launchpad.net/ubuntu/+source/flash-kernel/+bug/1160488/comments/3
or you do need a boot files update solution, and then if you opted to pull flash-kernel you should call its logic to pick which extra packages are needed for your platform.

This aint very convenient, but it's the current design; it's shared with Debian and it's an area where we're trying to avoid divergence (installer stuff).

[Scott Moser (smoser)]

Why couldn't whatever logic is used to decide "should I install flash-kernel" instead be used at runtime?

[Loïc Minier (lool)]

I dont understand your last point; I dont know how flash-kernel got pulled/called in the specific use case of this bug. This is what needs to be fixed to fit with the current approach: flash-kernel should not be installed since you dont really want the boot stuff to be installed, or if it is indeed needed then other packages ought to be pulled at the same time.

flash-kernel should only get called by matters of initramfs scripts or /etc/ bootloader setup scripts both shipped by the flash-kernel package itself.

[Scott Moser (smoser)]

deciding to install a package should not *break* other packages.
thats the problem we have.

the images are built to be generic. they don't have specific target when their built. and this is the only point where they have to know that target at "package selection time". Everything else just works.

The bug is easily recreated on a system that should not use flash-kernel:
 $ apt-get install flash-kernel
 $ upate-initramfs
 FAIL

thats a bug. installing 1 package should not break another.

[dann frazier (dannf)]

On Fri, Jul 18, 2014 at 10:31 AM, Scott Moser <email address hidden> wrote:
> deciding to install a package should not *break* other packages.
> thats the problem we have.
>
> the images are built to be generic. they don't have specific target
> when their built. and this is the only point where they have to know
> that target at "package selection time". Everything else just works.
>
> The bug is easily recreated on a system that should not use flash-kernel:
> $ apt-get install flash-kernel
> $ upate-initramfs
> FAIL
>
> thats a bug. installing 1 package should not break another.

Right, I'd agree that the hook shouldn't error out if a package
outside the dependency chain is not installed. Though big flashing
warnings that you're possibly about to make your system unbootable
would seem be appropriate.

[Oliver Grawert (ogra)]

please see teh other bugs ... make your environment contain "FLASH_KERNEL_SKIP=true" for all armhf devices and you should be fine (this is a var we share with debian)

[Scott Moser (smoser)]

Oliver, I can do that. thats easy.
the problem is I dont know *when* to do that and when not to.

I expect that update-initramfs would do that for me.

telling people "if you install a package, another package will break unless you set your environment to include FLASH_KERNEL_SKIP=true" is not really any better than telling them "if you install a package, another package will break".

Why can't update-initramfs (or flash-kernel, or whatever) decide at runtime if it needs to invoke flash-kernel.

Possibly allow the user to set a config file /etc/config/flash-kernel to override that logic, but by default do the right thing.