ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3g5: libvirt fails to check for duplicate address in hotplug xml and causes the guest to go to shutoff state
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Fix Released
|
High
|
Canonical Server | ||
libvirt (Ubuntu) |
Fix Released
|
Undecided
|
Ubuntu on IBM Power Systems Bug Triage | ||
Bionic |
Fix Released
|
Undecided
|
Unassigned | ||
Disco |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[Impact]
* Attaching a device to an already consumed address never was and will
work. But the current error handling might affect the device currently
at that address.
Think: "Attach A at 1, Attach B at 1 => Fail clean up slot 1"
* Upstream has a fix for this which better detects the bad case which
helps to avoid this becoming an issue.
[Test Case]
Create two images to test with:
qemu-img create -f qcow2 /var/lib/
qemu-img create -f qcow2 /var/lib/
Create a guest:
$ uvt-simplestrea
$ uvt-kvm create --password ubuntu tests-scsi-hotplug arch=amd64 release=bionic label=daily
Then edit and restart the guest to use a scsi disk, add:
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<controller type='scsi' index='0' model='
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</controller>
Finally create a hotplug XML file with the second disk referring to the same address
cat > hpdisk.xml << EOF
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/
<target dev='sdb' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
EOF
Then try to attach this device to the guest
$ virsh attach-device tests-scsi-hotplug hpdisk.xml
Without the fix you will get (which has further negative implications, but is enough for the test):
root@b:~# virsh attach-device tests-scsi-hotplug hpdisk.xml
error: Failed to attach device from hpdisk.xml
error: internal error: unable to execute QEMU command 'device_add': Duplicate ID 'scsi0-0-0-0' for device
With the fix you will get:
root@b:~# virsh attach-device tests-scsi-hotplug hpdisk.xml
error: Failed to attach device from hpdisk.xml
error: Requested operation is not valid: Domain already contains a disk with that address
[Regression Potential]
* Attaching a device to an already consumed address already failed and
will continue to fail. So no use case will change significantly due to
the intended change.
The regression one could think of is that the duplicate check would
match for an allowed device attachment and block it, but from my review
I think that will not happen.
[Other Info]
* n/a
---
== Comment: #0 - INDIRA P. JOGA - 2018-05-24 09:32:53 ==
Problem Description:
===================
libvirt fails to check for duplicate address in hotplug xml & error out before even trying to hotplug anything.
Steps to re-create:
1. boslcp4 is up with BMC:1.20 & PNOR: 20180420 levels
2. Guest is up with kernel
root@boslcp3g5:~# uname -a
Linux boslcp3g5 4.4.0-122-generic #146-Ubuntu SMP Mon Apr 23 15:33:25 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
root@boslcp3g5:~# uname -r
4.4.0-122-generic
3. boslcp3g5 guest is running with LTP run (45 hours) & stress-ng aio class ( 15 minutes).
4. Hotplug xml
root@boslcp3:/home# cat hp-disk.xml
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/
<target dev='sdd' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='2'/>
</disk>
5.Tried hotplug with duplicate address
root@boslcp3:/home# virsh attach-device boslcp3g5 hp-disk.xml --live
error: Failed to attach device from hp-disk.xml
error: internal error: unable to execute QEMU command 'device_add': Duplicate ID 'scsi0-0-0-2' for device
root@boslcp3:/home#
root@boslcp3:/home# virsh list --all
Id Name State
-------
3 boslcp3g2 running
4 boslcp3g3 running
7 boslcp3g4 running
- boslcp3g1 shut off
- boslcp3g5 shut off
6. Libvirt fails to check the duplicate address in hotplug xml & error before even trying to hotplug anything
== Comment: #5 - Daniel Henrique Barboza - 2019-03-26 15:43:55 ==
The fix for this bug was pushed upstream:
commit f1d6585300001c7
Author: Daniel Henrique Barboza <email address hidden>
Date: Fri Mar 15 18:06:45 2019 -0300
domain_conf: check device address before attach
After this patch, Libvirt is checking for duplicated address before trying to do hotplug operations.
tags: | added: architecture-ppc64le bugnameltc-168186 severity-high targetmilestone-inin--- |
Changed in ubuntu: | |
assignee: | nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) |
affects: | ubuntu → libvirt (Ubuntu) |
Changed in ubuntu-power-systems: | |
importance: | Undecided → High |
assignee: | nobody → Canonical Server Team (canonical-server) |
Changed in ubuntu-power-systems: | |
status: | New → Triaged |
description: | updated |
Changed in ubuntu-power-systems: | |
status: | Triaged → In Progress |
Changed in ubuntu-power-systems: | |
status: | In Progress → Fix Committed |
tags: |
added: targetmilestone-inin18045 removed: targetmilestone-inin--- |
Changed in ubuntu-power-systems: | |
status: | Fix Committed → Fix Released |
Thanks for the bug,
the commit is in libvirt 5.2 and therefore in Eoan already.
I'm working on a few other libvirt changes that I'd like to group into one SRU.
So give me a few days to check if the others are ready in time (uploaded together) or not (then this will most likely be uploaded soon).