subiquity

curtin fails to install the kernel on arm64 and s390x

Bug #1856038 reported by Paride Legovini on 2019-12-11

This bug affects 2 people

	Status	Importance	Assigned to
Ubuntu on IBM z Systems	Fix Released	Critical	Unassigned
curtin	Fix Released	Undecided	Unassigned
subiquity	Fix Released	Undecided	Unassigned

Bug Description

Subiquity fails to install the kernel when installing Focal on arm64. The tarball of /var/log of the installer system is attached. From /var/log/curtin/install.log:

        Processing triggers for flash-kernel (3.98ubuntu8) ...
        dpkg: error processing package flash-kernel (--configure):
         installed flash-kernel package post-installation script subprocess returned error exit status 2
        Errors were encountered while processing:
         flash-kernel
        E: Sub-process /usr/bin/dpkg returned an error code (1)
        Running command ['udevadm', 'settle'] with allowed return codes [0] (capture=False)
        TIMED subp(['udevadm', 'settle']): 0.093
        Running command ['umount', '/target/sys'] with allowed return codes [0] (capture=False)
        Running command ['umount', '/target/proc'] with allowed return codes [0] (capture=False)
        Running command ['umount', '/target/dev'] with allowed return codes [0] (capture=False)
        finish: cmd-install/stage-curthooks/builtin/cmd-curthooks/installing-kernel: FAIL: installing kernel
        finish: cmd-install/stage-curthooks/builtin/cmd-curthooks: FAIL: curtin command curthooks
        Traceback (most recent call last):
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/main.py", line 202, in main
            ret = args.func(args)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1574, in curthooks
            builtin_curthooks(cfg, target, state)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1424, in builtin_curthooks
            install_kernel(cfg, target)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 351, in install_kernel
            distro.install_packages([kernel_package], target=target)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/distro.py", line 375, in install_packages
            env=env, allow_daemons=allow_daemons)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/distro.py", line 274, in run_apt_command
            return inchroot.subp(cmd, env=env)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/util.py", line 697, in subp
            return subp(*args, **kwargs)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/util.py", line 275, in subp
            return _subp(*args, **kwargs)
          File "/snap/subiquity/1361/lib/python3.6/site-packages/curtin/util.py", line 141, in _subp
            cmd=args)
        curtin.util.ProcessExecutionError: Unexpected error while running command.
        Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
        Exit code: 100
        Reason: -
        Stdout: ''
        Stderr: ''
        Unexpected error while running command.
        Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
        Exit code: 100

Tags:

Related branches

~raharper/curtin:fix/initramfs-only-once-x86-only

Merged into curtin:master

Chad Smith: Approve on 2019-12-16

Michael Hudson-Doyle: Approve on 2019-12-16

Server Team CI bot: Approve (continuous-integration) on 2019-12-16

Revision history for this message

Paride Legovini (paride) wrote on 2019-12-11:

arm64_var_log.tar.gz Edit (486.8 KiB, application/x-tar)

tags:

added: rls-ff-incoming

Revision history for this message

Paride Legovini (paride) wrote on 2019-12-11: Re: subiquity fails to install the kernel on arm64 and s390x

s390x_install.log Edit (55.9 KiB, text/plain)

The same problem seems to affect s390x (see attachment).

summary:

- subiquity fails to install the kernel on arm64
+ subiquity fails to install the kernel on arm64 and s390x

Revision history for this message

Paride Legovini (paride) wrote on 2019-12-13:

The root of the problem seems to be:

[ 267.086787] cloud-init[1461]: Processing triggers for linux-image-4.4.0-170-generic (4.4.0-170.199) ...
[ 267.410038] cloud-init[1461]: /etc/kernel/postinst.d/zz-flash-kernel:
[ 267.411989] cloud-init[1461]: flash-kernel: deferring update (trigger activated)
[ 267.423883] cloud-init[1461]: Processing triggers for flash-kernel (3.0~rc.4ubuntu62.2) ...
[ 267.563889] cloud-init[1461]: Can't find /boot/vmlinuz-4.4.0-170-generic or /boot/initrd.img-4.4.0-170-generic
[ 267.579645] cloud-init[1461]: dpkg: error processing package flash-kernel (--configure):
[ 267.591371] cloud-init[1461]: subprocess installed post-installation script returned error exit status 1
[ 267.596234] cloud-init[1461]: Errors were encountered while processing:
[ 267.599077] cloud-init[1461]: flash-kernel

Likely to be caused by this commit:

https://git.launchpad.net/curtin/commit/?id=04adfd9a83964ccec944b0f015143ea3b5b21473

Possible workaround: restrict the "run update-initramfs once" logic to arm64 until we find what goes wrong on arm64 and s390x.

Frank Heimes (fheimes) on 2019-12-16

tags:

added: s390x

Revision history for this message

Frank Heimes (fheimes) wrote on 2019-12-16:

/var/log and /var/crash Edit (609.3 KiB, application/x-tar)

After updating my local installation environment to the latest daily image (timestamp Dec 15th 2019): "Ubuntu-Server 20.04 LTS _Focal Fossa_ - Alpha s390x (20191215)"
that includes subiquity 19.12.1 I also ran into the issue that's reported here.
Please see the attached tgz for /var/log and /var/crash.

tags:

added: installer

Revision history for this message

Frank Heimes (fheimes) wrote on 2019-12-16:

In addition adding a screenshot from the last installer screen:

                             [ View full log ]
                             [ View error report ]
                             [ Reboot Now ]

After after selecting "Restart the Installer" the installation didn't succeeded,
hence with this problem successful installations aren't possible anymore!

In addition adding a screenshot from the last installer screen:

================================================================================
  An error occurred during installation                               [ Help ]  
================================================================================
  ┌────────────────────────── An error has occurred ─────────────────────────┐  
  │        acquiring and extracting image from cp:///media/filesystem       ^│  
  │  configuring installed system                                            │  
  │    running '/snap/bin/subiquity.subiquity-configure-run'                 │  
  │    running '/snap/bin/subiquity.subiquity-configure-apt                  │  
  │/snap/subiquity/1359/usr/bin/python3 true'                                │  
  │      curtin command apt-config                                           │  
  │      curtin command in-target                                            │  
  │    running 'curtin curthooks'                                           ││  
  │      curtin command curthooks                                           ││  
  │        configuring apt configuring apt                                  ││  
  │        installing missing packages                                      ││  
  │        configuring iscsi service                                        ││  
  │        configuring raid (mdadm) service                                 ││  
  │        installing kernel                                                v│  
  └──────────────────────────────────────────────────────────────────────────┘  
                                                                                
                             [ View full log     ]                              
                             [ View error report ]                              
                             [ Reboot Now        ]

After after selecting "Restart the Installer" the installation didn't succeeded,
hence with this problem successful installations aren't possible anymore!

Frank Heimes (fheimes) on 2019-12-16

tags:	added: req4focal
Changed in ubuntu-z-systems:
importance:	Undecided → Critical

Frank Heimes (fheimes) on 2019-12-16

Changed in ubuntu-z-systems:
status:	New → In Progress

Revision history for this message

Server Team CI bot (server-team-bot) wrote on 2019-12-16:

This bug is fixed with commit b022ed42 to curtin on branch master.
To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=b022ed42

Changed in curtin:
status:	New → Fix Committed

Frank Heimes (fheimes) on 2019-12-17

Changed in ubuntu-z-systems:
status:	In Progress → Fix Committed

Paride Legovini (paride) on 2019-12-17

summary:	- subiquity fails to install the kernel on arm64 and s390x + curtin fails to install the kernel on arm64 and s390x
Changed in subiquity:
status:	New → Invalid

Revision history for this message

Paride Legovini (paride) wrote on 2019-12-19:

installer logs Edit (1.2 MiB, application/x-tar)

I tried to install on arm64 with subiquity 1415 (which includes curtin commit b022ed42), but it still fails:

--------------------------------
Processing triggers for flash-kernel (3.98ubuntu8) ...
dpkg: error processing package flash-kernel (--configure):
installed flash-kernel package post-installation script subprocess returned error exit status 2
Errors were encountered while processing:
flash-kernel
E: Sub-process /usr/bin/dpkg returned an error code (1)
Running command ['udevadm', 'settle'] with allowed return codes [0] (capture=False)
TIMED subp(['udevadm', 'settle']): 0.139
Running command ['umount', '/target/sys'] with allowed return codes [0] (capture=False)
Running command ['umount', '/target/proc'] with allowed return codes [0] (capture=False)
Running command ['umount', '/target/dev'] with allowed return codes [0] (capture=False)
finish: cmd-install/stage-curthooks/builtin/cmd-curthooks/installing-kernel: FAIL: installing kernel
finish: cmd-install/stage-curthooks/builtin/cmd-curthooks: FAIL: curtin command curthooks
Traceback (most recent call last):
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/main.py", line 202, in main
    ret = args.func(args)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1592, in curthooks
    builtin_curthooks(cfg, target, state)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1443, in builtin_curthooks
    install_kernel(cfg, target)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 369, in install_kernel
    distro.install_packages([kernel_package], target=target)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/distro.py", line 375, in install_packages
    env=env, allow_daemons=allow_daemons)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/distro.py", line 274, in run_apt_command
    return inchroot.subp(cmd, env=env)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 697, in subp
    return subp(*args, **kwargs)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 275, in subp
    return _subp(*args, **kwargs)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 141, in _subp
    cmd=args)
curtin.util.ProcessExecutionError: Unexpected error while running command.
Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
Exit code: 100
--------------------------------

The full installer logs are attached.

I tried to install on arm64 with subiquity 1415 (which includes curtin commit b022ed42), but it still fails:

--------------------------------
Processing triggers for flash-kernel (3.98ubuntu8) ...
dpkg: error processing package flash-kernel (--configure):
 installed flash-kernel package post-installation script subprocess returned error exit status 2
Errors were encountered while processing:
 flash-kernel
E: Sub-process /usr/bin/dpkg returned an error code (1)
Running command ['udevadm', 'settle'] with allowed return codes [0] (capture=False)
TIMED subp(['udevadm', 'settle']): 0.139
Running command ['umount', '/target/sys'] with allowed return codes [0] (capture=False)
Running command ['umount', '/target/proc'] with allowed return codes [0] (capture=False)
Running command ['umount', '/target/dev'] with allowed return codes [0] (capture=False)
finish: cmd-install/stage-curthooks/builtin/cmd-curthooks/installing-kernel: FAIL: installing kernel
finish: cmd-install/stage-curthooks/builtin/cmd-curthooks: FAIL: curtin command curthooks
Traceback (most recent call last):
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/main.py", line 202, in main
    ret = args.func(args)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1592, in curthooks
    builtin_curthooks(cfg, target, state)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 1443, in builtin_curthooks
    install_kernel(cfg, target)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/commands/curthooks.py", line 369, in install_kernel
    distro.install_packages([kernel_package], target=target)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/distro.py", line 375, in install_packages
    env=env, allow_daemons=allow_daemons)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/distro.py", line 274, in run_apt_command
    return inchroot.subp(cmd, env=env)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 697, in subp
    return subp(*args, **kwargs)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 275, in subp
    return _subp(*args, **kwargs)
  File "/snap/subiquity/1415/lib/python3.6/site-packages/curtin/util.py", line 141, in _subp
    cmd=args)
curtin.util.ProcessExecutionError: Unexpected error while running command.
Command: ['unshare', '--fork', '--pid', '--', 'chroot', '/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'linux-generic']
Exit code: 100
--------------------------------

The full installer logs are attached.

Changed in subiquity:
status:	Invalid → New

Paride Legovini (paride) on 2019-12-19

Changed in ubuntu-z-systems:
status:	Fix Committed → New
Changed in curtin:
status:	Fix Committed → New

Revision history for this message

Paride Legovini (paride) wrote on 2019-12-19:

On s390x it's fixed, see:

https://platform-qa-jenkins.ubuntu.com/view/server-f/job/ubuntu-focal-live-server-s390x-smoke-default/49/

https://jenkins.ubuntu.com/server/view/iso/job/isotest-focal-s390x/17/

I marked the task as "Fix Released".

Changed in ubuntu-z-systems:
status:	New → Fix Released

Revision history for this message

Frank Heimes (fheimes) wrote on 2019-12-19:

I currently do not run in the above kernel install problem while doing an LPAR installation (well, I can get that far), but ran into another severe problem that I reported here in LP 1857042
https://bugs.launchpad.net/curtin/+bug/1857042

Paride Legovini (paride) on 2020-01-10

Changed in curtin:
status:	New → Fix Released
Changed in subiquity:
status:	New → Fix Released

Revision history for this message

Paride Legovini (paride) wrote on 2020-01-22:

#10

For some reason I don't fully understand an older revision of subiquity is included in the latest ISO images, and the fix to this bug is not included anymore. This is certainly a temporary thing and the fix we have *is* valid, I'm however reopening this bug to better keep track of progress.

Changed in curtin:
status:	Fix Released → New
Changed in subiquity:
status:	Fix Released → New

Revision history for this message

Paride Legovini (paride) wrote on 2020-01-22:

#11

Reason for the regression: the subiquity snap in the stable/ubuntu-20.04 channel expired. It has to be published again and the images rebuilt with it.

Paride Legovini (paride) on 2020-01-24

Changed in curtin:
status:	New → Triaged
Changed in subiquity:
status:	New → Triaged

Revision history for this message

Dan Watkins (oddbloke) wrote on 2020-01-27:

#12

Moving this back to Fix Released for curtin, as the issue is still fixed in trunk.

Changed in curtin:
status:	Triaged → Fix Released

Dimitri John Ledkov (xnox) on 2020-04-01

Changed in subiquity:
status:	Triaged → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.