curtin fails to install the kernel on arm64 and s390x

Bug #1856038 reported by Paride Legovini on 2019-12-11
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Critical
Unassigned
curtin
Undecided
Unassigned
subiquity
Undecided
Unassigned

Bug Description

Subiquity fails to install the kernel when installing Focal on arm64. The tarball of /var/log of the installer system is attached. From /var/log/curtin/install.log:

        Processing triggers for flash-kernel (3.98ubuntu8) ...
        dpkg: error processing package flash-kernel (--configure):
         installed flash-kernel package post-installation script subprocess returned error exit status 2
        Errors were encountered while processing:
         flash-kernel
        E: Sub-process /usr/bin/dpkg returned an error code (1)
        Running command ['udevadm', 'settle'] with allowed return codes [0] (capture=False)
        TIMED subp(['udevadm', 'settle']): 0.093
        Running command ['umount', '/target/sys'] with allowed return codes [0] (capture=False)
        Running command ['umount', '/target/proc'] with allowed return codes [0] (capture=False)
        Running command ['umount', '/target/dev'] with allowed return codes [0] (capture=False)
        finish: cmd-install/stage-curthooks/builtin/cmd-curthooks/installing-kernel: FAIL: installing kernel
        finish: cmd-install/stage-curthooks/builtin/cmd-curthooks: FAIL: curtin command curthooks
        Traceback (most recent call last):
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/main.py", line 202, in main
            ret = args.func(args)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1574, in curthooks
            builtin_curthooks(cfg, target, state)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1424, in builtin_curthooks
            install_kernel(cfg, target)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 351, in install_kernel
            distro.install_packages([kernel_package], target=target)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/distro.py", line 375, in install_packages
            env=env, allow_daemons=allow_daemons)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/distro.py", line 274, in run_apt_command
            return inchroot.subp(cmd, env=env)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/util.py", line 697, in subp
            return subp(*args, **kwargs)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/util.py", line 275, in subp
            return _subp(*args, **kwargs)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/util.py", line 141, in _subp
            cmd=args)
        curtin.util.ProcessExecutionError: Unexpected error while running command.
        Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
        Exit code: 100
        Reason: -
        Stdout: ''
        Stderr: ''
        Unexpected error while running command.
        Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
        Exit code: 100

Related branches

Paride Legovini (legovini) wrote :
tags: added: rls-ff-incoming

The same problem seems to affect s390x (see attachment).

summary: - subiquity fails to install the kernel on arm64
+ subiquity fails to install the kernel on arm64 and s390x
Paride Legovini (legovini) wrote :

The root of the problem seems to be:

[ 267.086787] cloud-init[1461]: Processing triggers for linux-image-4.4.0-170-generic (4.4.0-170.199) ...
[ 267.410038] cloud-init[1461]: /etc/kernel/postinst.d/zz-flash-kernel:
[ 267.411989] cloud-init[1461]: flash-kernel: deferring update (trigger activated)
[ 267.423883] cloud-init[1461]: Processing triggers for flash-kernel (3.0~rc.4ubuntu62.2) ...
[ 267.563889] cloud-init[1461]: Can't find /boot/vmlinuz-4.4.0-170-generic or /boot/initrd.img-4.4.0-170-generic
[ 267.579645] cloud-init[1461]: dpkg: error processing package flash-kernel (--configure):
[ 267.591371] cloud-init[1461]: subprocess installed post-installation script returned error exit status 1
[ 267.596234] cloud-init[1461]: Errors were encountered while processing:
[ 267.599077] cloud-init[1461]: flash-kernel

Likely to be caused by this commit:

https://git.launchpad.net/curtin/commit/?id=04adfd9a83964ccec944b0f015143ea3b5b21473

Possible workaround: restrict the "run update-initramfs once" logic to arm64 until we find what goes wrong on arm64 and s390x.

Frank Heimes (fheimes) on 2019-12-16
tags: added: s390x
Frank Heimes (fheimes) wrote :

After updating my local installation environment to the latest daily image (timestamp Dec 15th 2019): "Ubuntu-Server 20.04 LTS _Focal Fossa_ - Alpha s390x (20191215)"
that includes subiquity 19.12.1 I also ran into the issue that's reported here.
Please see the attached tgz for /var/log and /var/crash.

tags: added: installer
Frank Heimes (fheimes) wrote :

In addition adding a screenshot from the last installer screen:

================================================================================
  An error occurred during installation [ Help ]
================================================================================
  ┌────────────────────────── An error has occurred ─────────────────────────┐
  │ acquiring and extracting image from cp:///media/filesystem ^│
  │ configuring installed system │
  │ running '/snap/bin/subiquity.subiquity-configure-run' │
  │ running '/snap/bin/subiquity.subiquity-configure-apt │
  │/snap/subiquity/1359/usr/bin/python3 true' │
  │ curtin command apt-config │
  │ curtin command in-target │
  │ running 'curtin curthooks' ││
  │ curtin command curthooks ││
  │ configuring apt configuring apt ││
  │ installing missing packages ││
  │ configuring iscsi service ││
  │ configuring raid (mdadm) service ││
  │ installing kernel v│
  └──────────────────────────────────────────────────────────────────────────

                             [ View full log ]
                             [ View error report ]
                             [ Reboot Now ]

After after selecting "Restart the Installer" the installation didn't succeeded,
hence with this problem successful installations aren't possible anymore!

Frank Heimes (fheimes) on 2019-12-16
tags: added: req4focal
Changed in ubuntu-z-systems:
importance: Undecided → Critical
Frank Heimes (fheimes) on 2019-12-16
Changed in ubuntu-z-systems:
status: New → In Progress

This bug is fixed with commit b022ed42 to curtin on branch master.
To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=b022ed42

Changed in curtin:
status: New → Fix Committed
Frank Heimes (fheimes) on 2019-12-17
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
summary: - subiquity fails to install the kernel on arm64 and s390x
+ curtin fails to install the kernel on arm64 and s390x
Changed in subiquity:
status: New → Invalid
Paride Legovini (legovini) wrote :

I tried to install on arm64 with subiquity 1415 (which includes curtin commit b022ed42), but it still fails:

--------------------------------
Processing triggers for flash-kernel (3.98ubuntu8) ...
dpkg: error processing package flash-kernel (--configure):
 installed flash-kernel package post-installation script subprocess returned error exit status 2
Errors were encountered while processing:
 flash-kernel
E: Sub-process /usr/bin/dpkg returned an error code (1)
Running command ['udevadm', 'settle'] with allowed return codes [0] (capture=False)
TIMED subp(['udevadm', 'settle']): 0.139
Running command ['umount', '/target/sys'] with allowed return codes [0] (capture=False)
Running command ['umount', '/target/proc'] with allowed return codes [0] (capture=False)
Running command ['umount', '/target/dev'] with allowed return codes [0] (capture=False)
finish: cmd-install/stage-curthooks/builtin/cmd-curthooks/installing-kernel: FAIL: installing kernel
finish: cmd-install/stage-curthooks/builtin/cmd-curthooks: FAIL: curtin command curthooks
Traceback (most recent call last):
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/main.py", line 202, in main
    ret = args.func(args)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1592, in curthooks
    builtin_curthooks(cfg, target, state)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1443, in builtin_curthooks
    install_kernel(cfg, target)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 369, in install_kernel
    distro.install_packages([kernel_package], target=target)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/distro.py", line 375, in install_packages
    env=env, allow_daemons=allow_daemons)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/distro.py", line 274, in run_apt_command
    return inchroot.subp(cmd, env=env)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 697, in subp
    return subp(*args, **kwargs)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 275, in subp
    return _subp(*args, **kwargs)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 141, in _subp
    cmd=args)
curtin.util.ProcessExecutionError: Unexpected error while running command.
Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
Exit code: 100
--------------------------------

The full installer logs are attached.

Changed in subiquity:
status: Invalid → New
Changed in ubuntu-z-systems:
status: Fix Committed → New
Changed in curtin:
status: Fix Committed → New
Paride Legovini (legovini) wrote :
Changed in ubuntu-z-systems:
status: New → Fix Released
Frank Heimes (fheimes) wrote :

I currently do not run in the above kernel install problem while doing an LPAR installation (well, I can get that far), but ran into another severe problem that I reported here in LP 1857042
https://bugs.launchpad.net/curtin/+bug/1857042

Changed in curtin:
status: New → Fix Released
Changed in subiquity:
status: New → Fix Released
Paride Legovini (legovini) wrote :

For some reason I don't fully understand an older revision of subiquity is included in the latest ISO images, and the fix to this bug is not included anymore. This is certainly a temporary thing and the fix we have *is* valid, I'm however reopening this bug to better keep track of progress.

Changed in curtin:
status: Fix Released → New
Changed in subiquity:
status: Fix Released → New
Paride Legovini (legovini) wrote :

Reason for the regression: the subiquity snap in the stable/ubuntu-20.04 channel expired. It has to be published again and the images rebuilt with it.

Changed in curtin:
status: New → Triaged
Changed in subiquity:
status: New → Triaged

Moving this back to Fix Released for curtin, as the issue is still fixed in trunk.

Changed in curtin:
status: Triaged → Fix Released
Changed in subiquity:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers