ubuntu 20.04 pxe installation fails with no such file or directory /dev/disk/by-id exception

[Ivan Kurnosov]

The `configure_grub_debconf` function tries to find a by-id path of the boot device.

The problem is though - when I run ubuntu 20.04 live cd via PXE - there is no `/dev/disk/by-id` there are only

1) by-label
2) by-partuuid
3) by-path
4) by-uuid

[Ivan Kurnosov (zerkms)]

A note: the installation is happening in KVM and it's not a scsi device, but a `/dev/vda` by default.

[Ryan Harper (raharper)]

virtio devices have serials, but not by default in qemu/kvm. As a workaround, you can add a serial to your virtio disk, using qemu command line:

 -drive if=none,id=bootdrive,file=$BOOT,format=qcow2
 -device virtio-blk-pci,drive=bootdrive,serial="myserial"

Or if you use libvirt, in the guest xml:

    <disk type='file' device='disk'>
      <source file='/media/lacie-500-disk-2/virtual-machines/scratch5.img'/>
      <target dev='vda' bus='virtio'/>
      <serial>dan123virtio</serial>
    </disk>

[Ryan Harper (raharper)]

If possible, can you collect /var/log/curtin/install.log

[Ryan Harper (raharper)]

grub-multi-install is affected by this as well.

[A B (onceler21)]

I have the same problem when installing to a xen virtual machine. The curtin install.log is attached.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andre Ruiz (andre-ruiz)]

Just to confirm, in a kvm client machine, going to the configs, VirtIO Disk 1, Advanced Options, Serial Number and typing anything there works around the problem. Just tested it.

[Mike EKlund (meklund-launchpad)]

I got the same error via pxe and manual install on vmware. If I use the default lvm partitioning it works, if I use manual non-LVM partitioning it fails.

i.e. this works:

 storage:
    swap:
      size: 0
    config:
    - {ptable: gpt, path: /dev/sda, wipe: superblock, preserve: false, name: '', grub_device: true,
      type: disk, id: disk-sda}
    - {device: disk-sda, size: 1M, flag: bios_grub, number: 1, preserve: false,
      type: partition, id: partition-0}
    - {device: disk-sda, size: 1G , wipe: superblock, flag: '', number: 2,
      preserve: false, type: partition, id: partition-1}
    - {fstype: ext4, volume: partition-1, preserve: false, type: format, id: format-0}
    - {device: disk-sda, size: -1 , wipe: superblock, flag: '', number: 3,
      preserve: false, type: partition, id: partition-2}
    - name: ubuntu-vg
      devices: [partition-2]
      preserve: false
      type: lvm_volgroup
      id: lvm_volgroup-0
    - {name: ubuntu-lv, volgroup: lvm_volgroup-0, size: 16101933056B, preserve: false,
      type: lvm_partition, id: lvm_partition-0}
    - {fstype: ext4, volume: lvm_partition-0, preserve: false, type: format, id: format-1}
    - {device: format-1, path: /, type: mount, id: mount-1}
    - {device: format-0, path: /boot, type: mount, id: mount-0}

This does not:

  storage:
    swap:
      size: 0
    config:
    - {ptable: gpt, path: /dev/sda, wipe: superblock, preserve: false, name: '', grub_device: true,
      type: disk, id: disk-sda}
    - {device: disk-sda, size: 1M, flag: bios_grub, number: 1, preserve: false,
      type: partition, id: partition-0}
    - {device: disk-sda, size: 1G , wipe: superblock, flag: '', number: 2,
      preserve: false, type: partition, id: partition-1}
    - {fstype: ext4, volume: partition-1, preserve: false, type: format, id: format-0}
    - {device: disk-sda, size: -1 , wipe: superblock, flag: '', number: 3,
      preserve: false, type: partition, id: partition-2}
    - {fstype: ext4, volume: partition-2, preserve: false, type: format, id: format-1}
    - {device: format-1, path: /, type: mount, id: mount-1}
    - {device: format-0, path: /boot, type: mount, id: mount-0}

[Ryan Harper (raharper)]

Curtin is collecting by-id links to disk to populate grub2 's debconf value:

grub-pc grub-pc/install_devices

It's not clear to me (yet) if providing non persistent values (like /dev/sda) will work with

/usr/lib/grub/grub-multi-install

Even if it accepts non-by-id values; it's possible that on subsequent boots (with additional disks present) that /dev/sda might not point to the disk that has grub.

So we can change this from an error by falling back on the path value, but it's just not safe. Disks need persistent attributes (like serial) for persistent links to work as expected.

[Lee Trager (ltrager)]

As per Ryan's comment in LP:1891251 MAAS should ensure block devices are created with serial numbers. This is possible with libvirt however we may need LXD to add support for this.

[Server Team CI bot (server-team-bot)]

This bug is fixed with commit 4cb10353 to curtin on branch master.
To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=4cb10353

[Julian Andres Klode (juliank)]

I don't believe there is a grub task here, as grub can fall back to plain device names, so closing this task.