with last proposed libvirt/qemu update instances missing

[Mehdi Abaakouk]

Hi,

Last libvirt/qemu package from xenial-proposed are broken, all instances looks vanished while they still live in memory and xml files are present:

# virsh list --all
 Id Name State
----------------------------------------------------

# ls /etc/libvirt/qemu
00abf967-f985-4d68-9245-e3d9bff3f460.xml 235c211d-3b46-42ff-b292-7ffda07d878b.xml 6759c4eb-d64e-4ca5-87fc-2595083945bc.xml a90ca251-d739-4cbf-b8eb-f19462525d60.xml cd83e06e-cbda-4184-a2df-de4391bf1272.xml
0417fae0-ff11-4e7b-918d-bc63922ead2e.xml 273bef09-cc5f-4029-83ab-2f55ba7066b0.xml 68087a51-8f0a-4684-b295-0d4cbfaff732.xml a9e82e45-b943-484b-8cfc-5e9c1438cbd6.xml e2b63103-80ee-458c-99b6-18892b293ed0.xml
04ff3f22-a97c-4999-9c90-f757ef018e5a.xml 376cbb41-71c6-4797-a307-7393f8507114.xml 6c485d85-c662-43f0-a428-5e7d6ae3a63f.xml ab342214-4cd2-4c7d-b6dd-4b6835613827.xml f449d0ac-52df-4862-b8f5-9e70c97aa27d.xml
0d36ba8e-afec-4f17-906a-b79ed02a5164.xml 3fe0f3ae-0914-4803-8942-612164932203.xml 6c821a07-f7ee-447d-b0da-203d3141c354.xml af44642c-e46d-4b7e-80e2-e90c8077a7a3.xml networks
0e80eca0-5a5e-40ac-97e9-d210aeb6662d.xml 42c3c4b3-ef9c-4c94-b33a-fff17300ed8f.xml 82db5dce-2576-4a85-8e1f-f45b6199ddd6.xml b53a40b6-2288-4f9e-8206-1ad79f9036fb.xml
101e31d9-5080-4aa3-8c81-9035db21edc1.xml 47425151-5b9f-42dc-8048-512f39d21c43.xml 8b52b54c-2e65-4e4f-82d6-0ff8f3100688.xml c412a749-a2fa-4e59-89be-8ccfced11ae0.xml
1d5511c3-e8d9-432e-a29b-e2dd94e1a8a5.xml 51d00dea-7305-4e71-ae5d-ef8249c234fa.xml a22cc903-c061-48fe-9045-8399e086616b.xml c93900f4-b6b7-455d-88e3-86f373abb677.xml

# systemctl status libvirtd.service
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2018-02-10 14:28:04 CET; 14min ago
     Docs: man:libvirtd(8)
           http://libvirt.org
 Main PID: 12352 (libvirtd)
   CGroup: /system.slice/libvirtd.service
           ├─ 1985 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           ├─ 1986 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           ├─11231 qemu-system-x86_64 -enable-kvm -name guest=3fe0f3ae-0914-4803-8942-612164932203,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-3fe0f3ae-0914-480
           ├─11756 qemu-system-x86_64 -enable-kvm -name guest=a22cc903-c061-48fe-9045-8399e086616b,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-a22cc903-c061-48f
           ├─12106 qemu-system-x86_64 -enable-kvm -name guest=00abf967-f985-4d68-9245-e3d9bff3f460,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-21-00abf967-f985-4d
           ├─12207 qemu-system-x86_64 -enable-kvm -name guest=a9e82e45-b943-484b-8cfc-5e9c1438cbd6,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-30-a9e82e45-b943-48
           ├─12284 qemu-system-x86_64 -enable-kvm -name guest=6c485d85-c662-43f0-a428-5e7d6ae3a63f,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-6c485d85-c662-43f
           ├─12352 /usr/sbin/libvirtd
           ├─12450 qemu-system-x86_64 -enable-kvm -name guest=42c3c4b3-ef9c-4c94-b33a-fff17300ed8f,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-22-42c3c4b3-ef9c-4c
           ├─13125 qemu-system-x86_64 -enable-kvm -name guest=1d5511c3-e8d9-432e-a29b-e2dd94e1a8a5,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-1d5511c3-e8d9-432
           ├─13653 qemu-system-x86_64 -enable-kvm -name guest=0e80eca0-5a5e-40ac-97e9-d210aeb6662d,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-5-0e80eca0-5a5e-40a
           ├─13688 qemu-system-x86_64 -enable-kvm -name guest=e2b63103-80ee-458c-99b6-18892b293ed0,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-23-e2b63103-80ee-45
           ├─14233 qemu-system-x86_64 -enable-kvm -name guest=0417fae0-ff11-4e7b-918d-bc63922ead2e,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-24-0417fae0-ff11-4e
           ├─14355 qemu-system-x86_64 -enable-kvm -name guest=273bef09-cc5f-4029-83ab-2f55ba7066b0,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-6-273bef09-cc5f-402
           ├─14655 qemu-system-x86_64 -enable-kvm -name guest=c93900f4-b6b7-455d-88e3-86f373abb677,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-7-c93900f4-b6b7-455
           ├─14928 qemu-system-x86_64 -enable-kvm -name guest=376cbb41-71c6-4797-a307-7393f8507114,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-31-376cbb41-71c6-47
           ├─15190 qemu-system-x86_64 -enable-kvm -name guest=235c211d-3b46-42ff-b292-7ffda07d878b,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-235c211d-3b46-42f
           ├─15494 qemu-system-x86_64 -enable-kvm -name guest=a90ca251-d739-4cbf-b8eb-f19462525d60,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-25-a90ca251-d739-4c
           ├─15983 qemu-system-x86_64 -enable-kvm -name guest=af44642c-e46d-4b7e-80e2-e90c8077a7a3,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-9-af44642c-e46d-4b7
           ├─16302 qemu-system-x86_64 -enable-kvm -name guest=04ff3f22-a97c-4999-9c90-f757ef018e5a,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-04ff3f22-a97c-49
           ├─16693 qemu-system-x86_64 -enable-kvm -name guest=68087a51-8f0a-4684-b295-0d4cbfaff732,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-11-68087a51-8f0a-46
           ├─16826 qemu-system-x86_64 -enable-kvm -name guest=6759c4eb-d64e-4ca5-87fc-2595083945bc,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-26-6759c4eb-d64e-4c
           ├─17022 qemu-system-x86_64 -enable-kvm -name guest=47425151-5b9f-42dc-8048-512f39d21c43,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-12-47425151-5b9f-42
           ├─17766 qemu-system-x86_64 -enable-kvm -name guest=cd83e06e-cbda-4184-a2df-de4391bf1272,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-13-cd83e06e-cbda-41
           ├─18949 qemu-system-x86_64 -enable-kvm -name guest=b53a40b6-2288-4f9e-8206-1ad79f9036fb,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-14-b53a40b6-2288-4f
           ├─19321 qemu-system-x86_64 -enable-kvm -name guest=ab342214-4cd2-4c7d-b6dd-4b6835613827,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-15-ab342214-4cd2-4c
           ├─19701 qemu-system-x86_64 -enable-kvm -name guest=c412a749-a2fa-4e59-89be-8ccfced11ae0,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-16-c412a749-a2fa-4e
           ├─20130 qemu-system-x86_64 -enable-kvm -name guest=8b52b54c-2e65-4e4f-82d6-0ff8f3100688,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-8b52b54c-2e65-4e
           ├─20974 qemu-system-x86_64 -enable-kvm -name guest=f449d0ac-52df-4862-b8f5-9e70c97aa27d,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-18-f449d0ac-52df-48
           ├─21213 qemu-system-x86_64 -enable-kvm -name guest=0d36ba8e-afec-4f17-906a-b79ed02a5164,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-27-0d36ba8e-afec-4f
           ├─21365 qemu-system-x86_64 -enable-kvm -name guest=82db5dce-2576-4a85-8e1f-f45b6199ddd6,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-19-82db5dce-2576-4a
           ├─21818 qemu-system-x86_64 -enable-kvm -name guest=101e31d9-5080-4aa3-8c81-9035db21edc1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-20-101e31d9-5080-4a
           ├─23053 qemu-system-x86_64 -enable-kvm -name guest=51d00dea-7305-4e71-ae5d-ef8249c234fa,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-28-51d00dea-7305-4e
           └─24534 qemu-system-x86_64 -enable-kvm -name guest=6c821a07-f7ee-447d-b0da-203d3141c354,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-29-6c821a07-f7ee-44

Feb 10 14:30:31 g12 libvirtd[12352]: 12355: warning : virQEMUCapsLogProbeFailure:4441 : Failed to probe capabilities for /usr/bin/qemu-system-xtensa: internal error: unable to execute QEMU command 'query-cpu-defi
Feb 10 14:30:31 g12 libvirtd[12352]: 12355: error : qemuMonitorJSONCheckError:387 : internal error: unable to execute QEMU command 'query-cpu-definitions': The command query-cpu-definitions has not been found
Feb 10 14:30:31 g12 libvirtd[12352]: 12355: warning : virQEMUCapsLogProbeFailure:4441 : Failed to probe capabilities for /usr/bin/qemu-system-xtensaeb: internal error: unable to execute QEMU command 'query-cpu-de
Feb 10 14:30:31 g12 libvirtd[12352]: 12355: error : virCapabilitiesDomainDataLookupInternal:717 : invalid argument: could not find capabilities for arch=x86_64 domaintype=qemu
Feb 10 14:30:31 g12 libvirtd[12352]: 12355: error : virQEMUCapsGetDefaultVersion:2082 : internal error: Cannot find suitable emulator for x86_64
Feb 10 14:30:31 g12 libvirtd[12352]: 12353: error : virCapabilitiesDomainDataLookupInternal:717 : invalid argument: could not find capabilities for arch=x86_64 domaintype=qemu
Feb 10 14:30:31 g12 libvirtd[12352]: 12353: error : virQEMUCapsGetDefaultVersion:2082 : internal error: Cannot find suitable emulator for x86_64
Feb 10 14:30:31 g12 libvirtd[12352]: 12358: error : virCapabilitiesDomainDataLookupInternal:717 : invalid argument: could not find capabilities for arch=x86_64 domaintype=qemu
Feb 10 14:30:31 g12 libvirtd[12352]: 12358: error : virQEMUCapsGetDefaultVersion:2082 : internal error: Cannot find suitable emulator for x86_64
Feb 10 14:30:31 g12 libvirtd[12352]: 12352: error : virNetSocketReadWire:1801 : End of file while reading data: Input/output error

[Mehdi Abaakouk (sileht)]

I'm on ubuntu xenial with ocata cloud-archive and broken packages are:

ii libvirt-bin 2.5.0-3ubuntu5.6~cloud2 amd64 programs for the libvirt library
ii qemu-kvm 1:2.8+dfsg-3ubuntu2.9~cloud2 amd64 QEMU Full virtualization

[Mehdi Abaakouk (sileht)]

Downgrading libvirt and qemu packages to 2.5.0-3ubuntu5.6~cloud1 and 1:2.8+dfsg-3ubuntu2.9~cloud1 fix the issue.

[Corey Bryant (corey.bryant)]

Hi Mehdi,

Thanks for reporting this. Looking into it now.

Corey

[Corey Bryant (corey.bryant)]

SEGFAULT running qemu: sudo LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
:/sbin:/bin /usr/bin/qemu-system-x86_64 -S -no-user-config -nodefaults -nographic -machine none,accel=
kvm:tcg -qmp unix:/var/lib/libvirt/qemu/capabilities.monitor.sock,server,nowait -pidfile /var/lib/libv
irt/qemu/capabilities.pidfile -daemonize

Backtrace from coredump: https://paste.ubuntu.com/=TGCY65VYvD/

[Corey Bryant (corey.bryant)]

Seems to be failing on line 1898 at: https://paste.ubuntu.com/=syhPtZ5gkR/

[Corey Bryant (corey.bryant)]

A little more digging:

Looking at the host_x86_cpu_class_init stack frame:

(gdb) print host_cpudef
$4 = {name = 0x0, level = 0, xlevel = 0, vendor = "GenuineIntel", family = 6, model = 42, stepping = 1, features = {0 <repeats 19 times>}, model_id = 0x0}

(gdb) print host_cpudef.model
$16 = 42

SandyBridge and SandyBridge-IBRS are model 42

[Corey Bryant (corey.bryant)]

qemu 1:2.8+dfsg-3ubuntu2.9~cloud3 is building now in ocata-proposed.

[Corey Bryant (corey.bryant)]

Regression testing with tempest successful on ocata-proposed:

======
Totals
======
Ran: 102 tests in 1543.5582 sec.
 - Passed: 94
 - Skipped: 8
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 908.5919 sec.

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package qemu - 1:2.8+dfsg-3ubuntu2.9~cloud3
---------------

 qemu (1:2.8+dfsg-3ubuntu2.9~cloud3) xenial-ocata; urgency=medium
 .
   * d/p/CVE-2017-5715-1.patch: Align with Kilo approach to this patch as
     Pike approach caused SEGFAULT (LP: #1748622).
   * d/p/CVE-2017-5715-5.patch: Fix typo in stepping field for Haswell-IBRS.