Comment 0 for bug 1925211

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Hot-unplug of disks leaves broken block devices around in Hirsute

Repro:
# Prep a backing disk
$ sudo fallocate -l 100M /tmp/zfsbase1
$ sudo fallocate -l 100M /tmp/zfsbase2
$ sudo zpool create zfsmirrortest mirror /tmp/zfsbase1 /tmp/zfsbase2
$ sudo zfs create -V 20M zfsmirrortest/vol1
$ cat > disk-zfs.xml << EOF
      <disk type='block' device='disk'>
         <driver name='qemu' type='raw'/>
         <source dev='/dev/zvol/zfsmirrortest/vol1'/>
         <target dev='vdc' bus='virtio'/>
      </disk>
EOF
# Get a guest
$ uvt-kvm create --disk 5 --password=ubuntu h release=hirsute arch=s390x label=daily
$ uvt-kvm wait h release=hirsute arch=s390x label=daily
# Attach and Detach disk
$ virsh attach-device h disk-zfs.xml
Device attached successfully
$ virsh detach-device h disk-zfs.xml
Device detached successfully

From libvirts POV it is gone at this point
$ virsh domblklist h
 Target Source
------------------------------------------------------------------
 vda /var/lib/uvtool/libvirt/images/hirsute-2nd-zfs.qcow
 vdb /var/lib/uvtool/libvirt/images/hirsute-2nd-zfs-ds.qcow

But the guest thinks still it is present
$ uvt-kvm ssh --insecure hirsute-2nd-zfs lsblk
  ...
  vdc 252:32 0 20M 0 disk

This even remains a while after (not a race).

Any access to it in the guest will hang (as you'd expect of a non-existing blockdev)
4 0 1758 1739 20 0 12140 4800 - S+ pts/0 0:00 | \_ sudo mkfs.ext4 /dev/vdc
4 0 1759 1758 20 0 6924 1044 - D+ pts/0 0:00 | \_ mkfs.ext4 /dev/vdc

The result above was originally found with hirsute-guest@hirsute-host on s390x

I do NOT see the same with groovy-guest@hirsute-host on s390x
I DO see the same with hirsute-guest@groovy-host on s390x
  => Guest version dependent not Host/Hipervisor dependent
I DO see the same with ZFS disks AND LVM disks being added&removed
  => not type dependent
I do NOT see the same on x86.
  => Arch dependent ??

... the evidence slowly points towards an issue in the guest, damn we are so
close to release - but non-fully detaching disks are critical in my POV :-/

Filing this as-is for awareness, but certainly this will need more debugging.
Unsure where this is going to eventually I'll now file it for kernel/udev/systemd.
If there are any known issues/components that are related let me know please!