MAAS

Bug #1807505
Comment #5

Comment 5 for bug 1807505

Revision history for this message

Andres Rodriguez (andreserl) wrote on 2018-12-08: Re: [Bug 1807505] Re: [2.5rc2] serial of a device is not populated for virtio-blk devices

Also, please test disk creating virtual
Machines from MAAS as that is now our supported path for KVM in MAAS!

On Sat, Dec 8, 2018 at 11:03 AM Andres Rodriguez <email address hidden>
wrote:

> The reason for this is that SERIAL is a made up name. It’s not the actual
> serial of a device. The SERIAL_SHORT is the actual serial name.
>
> I think MAAS already creates serial for the devices it creates, I would
> have to check, but I don’t think we can rely on using SERIAL because it’s
> jot the actual serial of the disk, it is again, just a made up name
> assigned from the software perspective.
>
>
>
> On Sat, Dec 8, 2018 at 10:35 AM Dmitrii Shcherbakov <
> <email address hidden>> wrote:
>
>> ** Description changed:
>>
>> Created a domain with a number of different devices (virtio-blk, virtio-
>> scsi, sata, NVMe) and added <serial/> tag to all of them for testing of
>> lp:1735839. Querying udev, I can see that ID_SERIAL is set but MAAS does
>> not use it to populate the "serial" field during commissioning.
>> +
>> + The reason is that only ID_SERIAL_SHORT is collected by MAAS, not
>> + ID_SERIAL.
>>
>> https://paste.ubuntu.com/p/pM7Tx8Hgbw/
>>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2'/>
>> <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio0.qcow2'/>
>> <backingStore/>
>> <target dev='vda' bus='virtio'/>
>> <serial>disk2</serial>
>> <alias name='virtio-disk0'/>
>> <address type='pci' domain='0x0000' bus='0x06' slot='0x00'
>> function='0x0'/>
>> </disk>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2'/>
>> <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio1.qcow2'/>
>> <backingStore/>
>> <target dev='vdb' bus='virtio'/>
>> <serial>disk3</serial>
>> <alias name='virtio-disk1'/>
>> <address type='pci' domain='0x0000' bus='0x07' slot='0x00'
>> function='0x0'/>
>> </disk>
>>
>> ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vda |
>> grep ID_SERIAL
>> ID_SERIAL=disk2
>>
>> ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vdb |
>> grep ID_SERIAL
>> ID_SERIAL=disk3
>>
>> Looking at the full output I can see that in the udev output there is no
>> ID_SERIAL_SHORT for virtio devices:
>>
>> udevadm info --query=property --name /dev/vda
>> DEVLINKS=/dev/disk/by-path/virtio-pci-0000:06:00.0
>> /dev/disk/by-dname/vda /dev/disk/by-path/pci-0000:06:00.0
>> /dev/disk/by-id/virtio-disk2
>> DEVNAME=/dev/vda
>> DEVPATH=/devices/pci0000:00/0000:00:02.5/0000:06:00.0/virtio5/block/vda
>> DEVTYPE=disk
>> ID_PATH=pci-0000:06:00.0
>> ID_PATH_TAG=pci-0000_06_00_0
>> ID_SERIAL=disk2
>> MAJOR=252
>> MINOR=0
>> SUBSYSTEM=block
>> TAGS=:systemd:
>> USEC_INITIALIZED=1789047
>>
>> And ID_SERIAL_SHORT is something that MAAS collects in provisioning
>> scripts for block devices:
>>
>>
>> https://github.com/maas/maas/blob/2.5.0-rc2/src/provisioningserver/refresh/node_info_scripts.py#L513-L521
>> # Grab the device path, serial number, and sata connection.
>> UDEV_MAPPINGS = {
>> "DEVNAME": "PATH",
>> "DEVPATH": "DEVPATH",
>> "ID_SERIAL_SHORT": "SERIAL",
>>
>> As a result the serial field is empty for virtio devices (noticed first
>> by looking at the UI):
>>
>> https://paste.ubuntu.com/p/RzMk7r5ZGH/
>>
>> {
>> "firmware_version": null,
>> "name": "vda",
>> "filesystem": null,
>> "id_path": "/dev/vda",
>> "block_size": 2048,
>> "system_id": "xeqrrw",
>> # ...
>> "path": "/dev/disk/by-dname/vda",
>> # ...
>> "serial": "",
>>
>> ** Description changed:
>>
>> Created a domain with a number of different devices (virtio-blk, virtio-
>> scsi, sata, NVMe) and added <serial/> tag to all of them for testing of
>> lp:1735839. Querying udev, I can see that ID_SERIAL is set but MAAS does
>> not use it to populate the "serial" field during commissioning.
>>
>> The reason is that only ID_SERIAL_SHORT is collected by MAAS, not
>> - ID_SERIAL.
>> + ID_SERIAL and it is not present for virtio-blk devices (even if it is
>> + provided via QEMU).
>>
>> https://paste.ubuntu.com/p/pM7Tx8Hgbw/
>>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2'/>
>> <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio0.qcow2'/>
>> <backingStore/>
>> <target dev='vda' bus='virtio'/>
>> <serial>disk2</serial>
>> <alias name='virtio-disk0'/>
>> <address type='pci' domain='0x0000' bus='0x06' slot='0x00'
>> function='0x0'/>
>> </disk>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2'/>
>> <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio1.qcow2'/>
>> <backingStore/>
>> <target dev='vdb' bus='virtio'/>
>> <serial>disk3</serial>
>> <alias name='virtio-disk1'/>
>> <address type='pci' domain='0x0000' bus='0x07' slot='0x00'
>> function='0x0'/>
>> </disk>
>>
>> ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vda |
>> grep ID_SERIAL
>> ID_SERIAL=disk2
>>
>> ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vdb |
>> grep ID_SERIAL
>> ID_SERIAL=disk3
>>
>> Looking at the full output I can see that in the udev output there is no
>> ID_SERIAL_SHORT for virtio devices:
>>
>> udevadm info --query=property --name /dev/vda
>> DEVLINKS=/dev/disk/by-path/virtio-pci-0000:06:00.0
>> /dev/disk/by-dname/vda /dev/disk/by-path/pci-0000:06:00.0
>> /dev/disk/by-id/virtio-disk2
>> DEVNAME=/dev/vda
>> DEVPATH=/devices/pci0000:00/0000:00:02.5/0000:06:00.0/virtio5/block/vda
>> DEVTYPE=disk
>> ID_PATH=pci-0000:06:00.0
>> ID_PATH_TAG=pci-0000_06_00_0
>> ID_SERIAL=disk2
>> MAJOR=252
>> MINOR=0
>> SUBSYSTEM=block
>> TAGS=:systemd:
>> USEC_INITIALIZED=1789047
>>
>> And ID_SERIAL_SHORT is something that MAAS collects in provisioning
>> scripts for block devices:
>>
>>
>> https://github.com/maas/maas/blob/2.5.0-rc2/src/provisioningserver/refresh/node_info_scripts.py#L513-L521
>> # Grab the device path, serial number, and sata connection.
>> UDEV_MAPPINGS = {
>> "DEVNAME": "PATH",
>> "DEVPATH": "DEVPATH",
>> "ID_SERIAL_SHORT": "SERIAL",
>>
>> As a result the serial field is empty for virtio devices (noticed first
>> by looking at the UI):
>>
>> https://paste.ubuntu.com/p/RzMk7r5ZGH/
>>
>> {
>> "firmware_version": null,
>> "name": "vda",
>> "filesystem": null,
>> "id_path": "/dev/vda",
>> "block_size": 2048,
>> "system_id": "xeqrrw",
>> # ...
>> "path": "/dev/disk/by-dname/vda",
>> # ...
>> "serial": "",
>>
>> --
>> You received this bug notification because you are subscribed to MAAS.
>> https://bugs.launchpad.net/bugs/1807505
>>
>> Title:
>> [2.5rc2] serial of a device is not populated for virtio-blk devices
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/maas/+bug/1807505/+subscriptions
>>
>> Launchpad-Notification-Type: bug
>> Launchpad-Bug: product=maas; status=New; importance=Undecided;
>> assignee=None;
>> Launchpad-Bug-Information-Type: Public
>> Launchpad-Bug-Private: no
>> Launchpad-Bug-Security-Vulnerability: no
>> Launchpad-Bug-Commenters: dmitriis
>> Launchpad-Bug-Reporter: Dmitrii Shcherbakov (dmitriis)
>> Launchpad-Bug-Modifier: Dmitrii Shcherbakov (dmitriis)
>> Launchpad-Message-Rationale: Subscriber (MAAS)
>> Launchpad-Message-For: andreserl
>>
> --
> Andres Rodriguez (RoAkSoAx)
> Ubuntu Server Developer
> MSc. Telecom & Networking
> Systems Engineer
>
--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

Also, please test disk creating virtual
Machines from MAAS as that is now our supported path for KVM in MAAS!

On Sat, Dec 8, 2018 at 11:03 AM Andres Rodriguez <andreserl@ubuntu-pe.org>
wrote:

> The reason for this is that SERIAL is a made up name. It’s not the actual
> serial of a device. The SERIAL_SHORT is the actual serial name.
>
> I think MAAS already creates serial for the devices it creates, I would
> have to check, but I don’t think we can rely on using SERIAL because it’s
> jot the actual serial of the disk, it is again, just a made up name
> assigned from the software perspective.
>
>
>
> On Sat, Dec 8, 2018 at 10:35 AM Dmitrii Shcherbakov <
> 1807505@bugs.launchpad.net> wrote:
>
>> ** Description changed:
>>
>>   Created a domain with a number of different devices (virtio-blk, virtio-
>>   scsi, sata, NVMe) and added <serial/> tag to all of them for testing of
>>   lp:1735839. Querying udev, I can see that ID_SERIAL is set but MAAS does
>>   not use it to populate the "serial" field during commissioning.
>> +
>> + The reason is that only ID_SERIAL_SHORT is collected by MAAS, not
>> + ID_SERIAL.
>>
>>   https://paste.ubuntu.com/p/pM7Tx8Hgbw/
>>
>>       <disk type='file' device='disk'>
>>         <driver name='qemu' type='qcow2'/>
>>         <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio0.qcow2'/>
>>         <backingStore/>
>>         <target dev='vda' bus='virtio'/>
>>         <serial>disk2</serial>
>>         <alias name='virtio-disk0'/>
>>         <address type='pci' domain='0x0000' bus='0x06' slot='0x00'
>> function='0x0'/>
>>       </disk>
>>       <disk type='file' device='disk'>
>>         <driver name='qemu' type='qcow2'/>
>>         <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio1.qcow2'/>
>>         <backingStore/>
>>         <target dev='vdb' bus='virtio'/>
>>         <serial>disk3</serial>
>>         <alias name='virtio-disk1'/>
>>         <address type='pci' domain='0x0000' bus='0x07' slot='0x00'
>> function='0x0'/>
>>       </disk>
>>
>>   ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vda |
>> grep ID_SERIAL
>>   ID_SERIAL=disk2
>>
>>   ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vdb |
>> grep ID_SERIAL
>>   ID_SERIAL=disk3
>>
>>   Looking at the full output I can see that in the udev output there is no
>>   ID_SERIAL_SHORT for virtio devices:
>>
>>   udevadm info --query=property --name /dev/vda
>>   DEVLINKS=/dev/disk/by-path/virtio-pci-0000:06:00.0
>> /dev/disk/by-dname/vda /dev/disk/by-path/pci-0000:06:00.0
>> /dev/disk/by-id/virtio-disk2
>>   DEVNAME=/dev/vda
>>   DEVPATH=/devices/pci0000:00/0000:00:02.5/0000:06:00.0/virtio5/block/vda
>>   DEVTYPE=disk
>>   ID_PATH=pci-0000:06:00.0
>>   ID_PATH_TAG=pci-0000_06_00_0
>>   ID_SERIAL=disk2
>>   MAJOR=252
>>   MINOR=0
>>   SUBSYSTEM=block
>>   TAGS=:systemd:
>>   USEC_INITIALIZED=1789047
>>
>>   And ID_SERIAL_SHORT is something that MAAS collects in provisioning
>>   scripts for block devices:
>>
>>
>> https://github.com/maas/maas/blob/2.5.0-rc2/src/provisioningserver/refresh/node_info_scripts.py#L513-L521
>>       # Grab the device path, serial number, and sata connection.
>>       UDEV_MAPPINGS = {
>>           "DEVNAME": "PATH",
>>           "DEVPATH": "DEVPATH",
>>           "ID_SERIAL_SHORT": "SERIAL",
>>
>>   As a result the serial field is empty for virtio devices (noticed first
>>   by looking at the UI):
>>
>>   https://paste.ubuntu.com/p/RzMk7r5ZGH/
>>
>>           {
>>               "firmware_version": null,
>>               "name": "vda",
>>               "filesystem": null,
>>               "id_path": "/dev/vda",
>>               "block_size": 2048,
>>               "system_id": "xeqrrw",
>>   # ...
>>               "path": "/dev/disk/by-dname/vda",
>>   # ...
>>               "serial": "",
>>
>> ** Description changed:
>>
>>   Created a domain with a number of different devices (virtio-blk, virtio-
>>   scsi, sata, NVMe) and added <serial/> tag to all of them for testing of
>>   lp:1735839. Querying udev, I can see that ID_SERIAL is set but MAAS does
>>   not use it to populate the "serial" field during commissioning.
>>
>>   The reason is that only ID_SERIAL_SHORT is collected by MAAS, not
>> - ID_SERIAL.
>> + ID_SERIAL and it is not present for virtio-blk devices (even if it is
>> + provided via QEMU).
>>
>>   https://paste.ubuntu.com/p/pM7Tx8Hgbw/
>>
>>       <disk type='file' device='disk'>
>>         <driver name='qemu' type='qcow2'/>
>>         <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio0.qcow2'/>
>>         <backingStore/>
>>         <target dev='vda' bus='virtio'/>
>>         <serial>disk2</serial>
>>         <alias name='virtio-disk0'/>
>>         <address type='pci' domain='0x0000' bus='0x06' slot='0x00'
>> function='0x0'/>
>>       </disk>
>>       <disk type='file' device='disk'>
>>         <driver name='qemu' type='qcow2'/>
>>         <source
>> file='/mnt/sec1/libvirt-images/maas-vhost6-virtio1.qcow2'/>
>>         <backingStore/>
>>         <target dev='vdb' bus='virtio'/>
>>         <serial>disk3</serial>
>>         <alias name='virtio-disk1'/>
>>         <address type='pci' domain='0x0000' bus='0x07' slot='0x00'
>> function='0x0'/>
>>       </disk>
>>
>>   ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vda |
>> grep ID_SERIAL
>>   ID_SERIAL=disk2
>>
>>   ubuntu@maas-vhost6:~$ udevadm info --query=property --name /dev/vdb |
>> grep ID_SERIAL
>>   ID_SERIAL=disk3
>>
>>   Looking at the full output I can see that in the udev output there is no
>>   ID_SERIAL_SHORT for virtio devices:
>>
>>   udevadm info --query=property --name /dev/vda
>>   DEVLINKS=/dev/disk/by-path/virtio-pci-0000:06:00.0
>> /dev/disk/by-dname/vda /dev/disk/by-path/pci-0000:06:00.0
>> /dev/disk/by-id/virtio-disk2
>>   DEVNAME=/dev/vda
>>   DEVPATH=/devices/pci0000:00/0000:00:02.5/0000:06:00.0/virtio5/block/vda
>>   DEVTYPE=disk
>>   ID_PATH=pci-0000:06:00.0
>>   ID_PATH_TAG=pci-0000_06_00_0
>>   ID_SERIAL=disk2
>>   MAJOR=252
>>   MINOR=0
>>   SUBSYSTEM=block
>>   TAGS=:systemd:
>>   USEC_INITIALIZED=1789047
>>
>>   And ID_SERIAL_SHORT is something that MAAS collects in provisioning
>>   scripts for block devices:
>>
>>
>> https://github.com/maas/maas/blob/2.5.0-rc2/src/provisioningserver/refresh/node_info_scripts.py#L513-L521
>>       # Grab the device path, serial number, and sata connection.
>>       UDEV_MAPPINGS = {
>>           "DEVNAME": "PATH",
>>           "DEVPATH": "DEVPATH",
>>           "ID_SERIAL_SHORT": "SERIAL",
>>
>>   As a result the serial field is empty for virtio devices (noticed first
>>   by looking at the UI):
>>
>>   https://paste.ubuntu.com/p/RzMk7r5ZGH/
>>
>>           {
>>               "firmware_version": null,
>>               "name": "vda",
>>               "filesystem": null,
>>               "id_path": "/dev/vda",
>>               "block_size": 2048,
>>               "system_id": "xeqrrw",
>>   # ...
>>               "path": "/dev/disk/by-dname/vda",
>>   # ...
>>               "serial": "",
>>
>> --
>> You received this bug notification because you are subscribed to MAAS.
>> https://bugs.launchpad.net/bugs/1807505
>>
>> Title:
>>   [2.5rc2] serial of a device is not populated for virtio-blk devices
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/maas/+bug/1807505/+subscriptions
>>
>> Launchpad-Notification-Type: bug
>> Launchpad-Bug: product=maas; status=New; importance=Undecided;
>> assignee=None;
>> Launchpad-Bug-Information-Type: Public
>> Launchpad-Bug-Private: no
>> Launchpad-Bug-Security-Vulnerability: no
>> Launchpad-Bug-Commenters: dmitriis
>> Launchpad-Bug-Reporter: Dmitrii Shcherbakov (dmitriis)
>> Launchpad-Bug-Modifier: Dmitrii Shcherbakov (dmitriis)
>> Launchpad-Message-Rationale: Subscriber (MAAS)
>> Launchpad-Message-For: andreserl
>>
> --
> Andres Rodriguez (RoAkSoAx)
> Ubuntu Server Developer
> MSc. Telecom & Networking
> Systems Engineer
>
-- 
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer