Bug #1861369 “install failed crashed with CalledProcessError” : Bugs : The Ubuntu-power-systems project

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-01-30:

#1

CurrentDmesg.txt Edit (64.1 KiB, text/plain; charset="utf-8")
CurtinConfig.txt Edit (3.3 KiB, text/plain; charset="utf-8")
CurtinErrors.gz Edit (21.6 KiB, application/x-gzip)
CurtinLog.txt Edit (130.3 KiB, text/plain; charset="utf-8")
HookError_powerpc.txt Edit (393 bytes, text/plain; charset="utf-8")
InstallerLog.txt Edit (63.1 KiB, text/plain; charset="utf-8")
InstallerLogInfo.txt Edit (4.8 KiB, text/plain; charset="utf-8")
Lspci.txt Edit (46.5 KiB, text/plain; charset="utf-8")
Lsusb.txt Edit (570 bytes, text/plain; charset="utf-8")
ProbeData.txt Edit (46.2 KiB, text/plain; charset="utf-8")
ProcCpuinfoMinimal.txt Edit (1.3 KiB, text/plain; charset="utf-8")
ProcInterrupts.txt Edit (24.4 KiB, text/plain; charset="utf-8")
ProcModules.txt Edit (4.2 KiB, text/plain; charset="utf-8")
ProcStatus.txt Edit (1.1 KiB, text/plain; charset="utf-8")
Traceback.txt Edit (964 bytes, text/plain; charset="utf-8")
UdevDb.txt Edit (149.6 KiB, text/plain; charset="utf-8")

tags:	added: champagne
information type:	Private → Public

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-02-02:

#2

Download full text (4.6 KiB)

These lines stand out to me:

https://github.com/CanonicalLtd/subiquity/blob/436ee1d76ddad5362c25843f429b4346324c0b8f/subiquity/models/filesystem.py#L1368

    def render(self):
        config = {
            'storage': {
                'version': 1,
                'config': self._render_actions(),
                },
            }
        if not self._should_add_swapfile():
            config['swap'] = {'size': 0}
        if self.grub_install_device:
            dev = self.grub_install_device
            if dev.type == "partition":
                devpath = "{}{}".format(dev.device.path, dev._number)
            else:
                devpath = dev.path
            config['grub'] = {
                'install_devices': [devpath],
                }
        return config

https://github.com/CanonicalLtd/subiquity/blob/436ee1d76ddad5362c25843f429b4346324c0b8f/subiquity/controllers/filesystem.py#L390

    def _create_boot_partition(self, disk):
        bootloader = self.model.bootloader
        if bootloader == Bootloader.UEFI:
            part_size = UEFI_GRUB_SIZE_BYTES
            if UEFI_GRUB_SIZE_BYTES*2 >= disk.size:
                part_size = disk.size // 2
            log.debug('_create_boot_partition - adding EFI partition')
            part = self.create_partition(
                disk,
                dict(size=part_size, fstype='fat32', mount='/boot/efi'),
                flag="boot")
        elif bootloader == Bootloader.PREP:
            log.debug('_create_boot_partition - adding PReP partition')
            part = self.create_partition(
                disk,
                dict(size=PREP_GRUB_SIZE_BYTES, fstype=None, mount=None),
                # must be wiped or grub-install will fail
                wipe='zero',
                flag='prep')
            self.model.grub_install_device = part
        elif bootloader == Bootloader.BIOS:
            log.debug('_create_boot_partition - adding bios_grub partition')
            part = self.create_partition(
                disk,
                dict(size=BIOS_GRUB_SIZE_BYTES, fstype=None, mount=None),
                flag='bios_grub')
            self.model.grub_install_device = disk
        return part

https://github.com/CanonicalLtd/subiquity/blob/436ee1d76ddad5362c25843f429b4346324c0b8f/subiquity/controllers/filesystem.py#L613

    def make_boot_disk(self, new_boot_disk):
        boot_partition = None
        if self.model.bootloader == Bootloader.BIOS:
            install_dev = self.model.grub_install_device
            if install_dev:
                boot_partition = install_dev._potential_boot_partition()
        elif self.model.bootloader == Bootloader.UEFI:
            mount = self.model._mount_for_path("/boot/efi")
            if mount is not None:
                boot_partition = mount.device.volume
        elif self.model.bootloader == Bootloader.PREP:
            boot_partition = self.model.grub_install_device
        if boot_partition is not None:
            if boot_partition.preserve:
                if self.model.bootloader == Bootloader.PREP:
                    boot_partition.wipe = None
                elif self.model.bootloader == Bootloader.UEFI:
              ...

These lines stand out to me:

https://github.com/CanonicalLtd/subiquity/blob/436ee1d76ddad5362c25843f429b4346324c0b8f/subiquity/models/filesystem.py#L1368

def render(self):
        config = {
            'storage': {
                'version': 1,
                'config': self._render_actions(),
                },
            }
        if not self._should_add_swapfile():
            config['swap'] = {'size': 0}
        if self.grub_install_device:
            dev = self.grub_install_device
            if dev.type == "partition":
                devpath = "{}{}".format(dev.device.path, dev._number)
            else:
                devpath = dev.path
            config['grub'] = {
                'install_devices': [devpath],
                }
        return config

https://github.com/CanonicalLtd/subiquity/blob/436ee1d76ddad5362c25843f429b4346324c0b8f/subiquity/controllers/filesystem.py#L390

def _create_boot_partition(self, disk):
        bootloader = self.model.bootloader
        if bootloader == Bootloader.UEFI:
            part_size = UEFI_GRUB_SIZE_BYTES
            if UEFI_GRUB_SIZE_BYTES*2 >= disk.size:
                part_size = disk.size // 2
            log.debug('_create_boot_partition - adding EFI partition')
            part = self.create_partition(
                disk,
                dict(size=part_size, fstype='fat32', mount='/boot/efi'),
                flag="boot")
        elif bootloader == Bootloader.PREP:
            log.debug('_create_boot_partition - adding PReP partition')
            part = self.create_partition(
                disk,
                dict(size=PREP_GRUB_SIZE_BYTES, fstype=None, mount=None),
                # must be wiped or grub-install will fail
                wipe='zero',
                flag='prep')
            self.model.grub_install_device = part
        elif bootloader == Bootloader.BIOS:
            log.debug('_create_boot_partition - adding bios_grub partition')
            part = self.create_partition(
                disk,
                dict(size=BIOS_GRUB_SIZE_BYTES, fstype=None, mount=None),
                flag='bios_grub')
            self.model.grub_install_device = disk
        return part

https://github.com/CanonicalLtd/subiquity/blob/436ee1d76ddad5362c25843f429b4346324c0b8f/subiquity/controllers/filesystem.py#L613

def make_boot_disk(self, new_boot_disk):
        boot_partition = None
        if self.model.bootloader == Bootloader.BIOS:
            install_dev = self.model.grub_install_device
            if install_dev:
                boot_partition = install_dev._potential_boot_partition()
        elif self.model.bootloader == Bootloader.UEFI:
            mount = self.model._mount_for_path("/boot/efi")
            if mount is not None:
                boot_partition = mount.device.volume
        elif self.model.bootloader == Bootloader.PREP:
            boot_partition = self.model.grub_install_device
        if boot_partition is not None:
            if boot_partition.preserve:
                if self.model.bootloader == Bootloader.PREP:
                    boot_partition.wipe = None
                elif self.model.bootloader == Bootloader.UEFI:
                    self.delete_mount(boot_partition.fs().mount())
            else:
                boot_disk = boot_partition.device
                full = boot_disk.free_for_partitions == 0
                self.delete_partition(boot_partition)
                if full:
                    largest_part = max(
                        boot_disk.partitions(), key=lambda p: p.size)
                    largest_part.size += boot_partition.size
                if new_boot_disk.free_for_partitions < boot_partition.size:
                    largest_part = max(
                        new_boot_disk.partitions(), key=lambda p: p.size)
                    largest_part.size -= (
                        boot_partition.size -
                        new_boot_disk.free_for_partitions)
        if new_boot_disk._has_preexisting_partition():
            if self.model.bootloader == Bootloader.BIOS:
                self.model.grub_install_device = new_boot_disk
            elif self.model.bootloader == Bootloader.UEFI:
                part = new_boot_disk._potential_boot_partition()
                if part.fs() is None:
                    self.model.add_filesystem(part, 'fat32')
                self.model.add_mount(part.fs(), '/boot/efi')
            elif self.model.bootloader == Bootloader.PREP:
                part = new_boot_disk._potential_boot_partition()
                part.wipe = 'zero'
                self.model.grub_install_device = part
        else:
            new_boot_disk.preserve = False
            self._create_boot_partition(new_boot_disk)

Thanks

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-02-02:

#3

I think if my installer could go through the other branch of this if condition I'd probably have success:

            if dev.type == "partition":
                devpath = "{}{}".format(dev.device.path, dev._number)
            else:
                devpath = dev.path

Thanks

Revision history for this message

Frank Heimes (fheimes) wrote on 2020-03-26:

#4

Iirc the latest subiquity release (20.03) incl. fixes for such kind of block prob issues.
Would you mind giving the latest ISO a try so that we know if this particular issue is still valid?
Nevertheless, there is another issue open about NVMe devices that is probably not fixed yet: LP 1861655 - but let's go steps by step ...

Changed in ubuntu-power-systems:
importance:	Undecided → High
assignee:	nobody → Canonical Foundations Team (canonical-foundations)

Andrew Cloke (andrew-cloke) on 2020-03-27

tags:

added: installer

Frank Heimes (fheimes) on 2020-04-06

Changed in ubuntu-power-systems:
status:	New → Incomplete
Changed in subiquity:
status:	New → Incomplete

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-04-14:

#5

Hello Frank, I can't recall where exactly this failed any more, but at least now my install is failing in a different fashion:

https://bugs.launchpad.net/ubuntu/+source/subiquity/+bug/1872783

Thanks

Revision history for this message

Frank Heimes (fheimes) wrote on 2020-04-14:

#6

Closing project entry and setting it to Fix Released with todays ISO (Apr 14th).

Changed in ubuntu-power-systems:
status:	Incomplete → Fix Released

Revision history for this message

Dimitri John Ledkov (xnox) wrote on 2020-04-14:

#7

- {device: disk-nvme0n1, size: 8388608, wipe: zero, flag: prep, number: 1, preserve: false,
type: partition, id: partition-0}

seems to result in curtin doing

Command: ['sh', '-c', 'exec "$0" "$@" 2>&1', 'install-grub', '--os-family=debian', '/target', '/dev/nvme0n11']

which is incorrect, because `device: disk-nvme0n1` `number: 1` is clearly /dev/nvme0n1p1, not /dev/nvme0n11

Changed in curtin (Ubuntu):
status:	New → Triaged
importance:	Undecided → Critical

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-04-14:

#8

Frank, I'm sorry to say that bug 1872783 'masked' this bug for me. I couldn't reproduce 1872783 to get logs, but at the same time, found that this one is still happening.

Thanks

Apport retracing service (apport) on 2020-04-14

tags:

removed: need-duplicate-check

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-04-15:

#9

sarnold-power9.tar.gz Edit (246.1 KiB, application/x-tar)

Two .crash files, one trying to configure LVM, one not trying to configure LVM.

Frank Heimes (fheimes) on 2020-04-15

Changed in ubuntu-power-systems:
status:	Fix Released → Triaged

Revision history for this message

Frank Heimes (fheimes) wrote on 2020-04-15:

#10

[CanonicalLtd/subiquity] use curtin.block.partition_kname to calculate partition devpath (#706)
https://github.com/CanonicalLtd/subiquity/pull/706

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-04-15:

#11

I had success with https://launchpad.net/~mwhudson/+snap/subiquity-hack/+build/908976

Thanks Michael!

Revision history for this message

Michael Hudson-Doyle (mwhudson) wrote on 2020-04-15:

#12

Thanks for testing, at least this was easy in the end.

Changed in subiquity:
status:	Incomplete → Fix Committed
Changed in curtin (Ubuntu):
status:	Triaged → Invalid

Frank Heimes (fheimes) on 2020-04-15

Changed in ubuntu-power-systems:
status:	Triaged → Fix Committed

Revision history for this message

Frank Heimes (fheimes) wrote on 2020-04-20:

#13

Seth, did I interpret it correctly that you were not able to reproduce the issue with the latest ISO/installer and that it got fixed in between?

Revision history for this message

Seth Arnold (seth-arnold) wrote on 2020-04-21:

#14

Frank, Michael made available a private build of the installer in a separate snap channel, and it worked perfectly.

I don't know if this has made it into the 'release' channel of the installer snap, but it's very promising.

Thanks

Revision history for this message

Michael Hudson-Doyle (mwhudson) wrote on 2020-04-21:

#15

Yes, that fix landed. That said I'm just about to release another version which does this a different way -- testing the next image on ppc64el would be very useful!

Andrew Cloke (andrew-cloke) on 2020-04-27

Changed in subiquity:
status:	Fix Committed → Fix Released
Changed in ubuntu-power-systems:
status:	Fix Committed → Fix Released

Revision history for this message

Frank Heimes (fheimes) wrote on 2020-04-27:

#16

This is included in the GA version of 20.04 (released last Thursday), hence closing this ticket as Fix Released.

The Ubuntu-power-systems project

install failed crashed with CalledProcessError

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

	Status	Importance	Assigned to
The Ubuntu-power-systems project	Fix Released	High	Canonical Foundations Team
subiquity	Fix Released	Undecided	Unassigned
curtin (Ubuntu)	Invalid	Critical	Unassigned