14.04.5 safe-upgrade from QEMU 2.0.0+dfsg-2ubuntu1.42 to QEMU 2.0.0+dfsg-2ubuntu1.43 kernel panics amd64 VM guests

Bug #1776575 reported by wkitty42
274
This bug affects 3 people
Affects Status Importance Assigned to Milestone
qemu (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

on 2018 May 22, the following packages were installed on the Kubuntu Desktop 14.04.5 amd64 HOST system via "aptitude safe-upgrade"...

-rw-r--r-- 1 root root 167820 May 21 17:44 qemu_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 21730 May 21 17:44 qemu-keymaps_2.0.0+dfsg-2ubuntu1.42_all.deb
-rw-r--r-- 1 root root 7114 May 21 17:44 qemu-kvm_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 6096 May 21 17:44 qemu-system_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 1690746 May 21 17:44 qemu-system-arm_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 158472 May 21 17:44 qemu-system-common_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 2642646 May 21 17:44 qemu-system-mips_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 4928962 May 21 17:44 qemu-system-misc_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 2799884 May 21 17:44 qemu-system-ppc_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 1607216 May 21 17:44 qemu-system-sparc_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 1953488 May 21 17:44 qemu-system-x86_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 5473484 May 21 17:44 qemu-user_2.0.0+dfsg-2ubuntu1.42_amd64.deb
-rw-r--r-- 1 root root 411858 May 21 17:44 qemu-utils_2.0.0+dfsg-2ubuntu1.42_amd64.deb

on 2018 June 12, those same packages were upgraded their .43 release... again, on the HOST... diagnosis and recovery took place via #ubuntu on the freenode IRC channel...

SYMPTOMS:

1. all 64bit VM guests kernel panic on pivot after grub2 boot...
2. 32bit VM guests boot and operate as normal...
3. the VM used for diagnosis and recovery is running ubuntu server 14.04.5...
4. all *buntu systems, hosts and guests, are updated via "aptitude safe-upgrade" since installation...

RECOVERY:

downgraded all 13 installed QEMU related packages from .43 to .42...

all 64bit VM guests boot and operate as normal...
all 32bit VM guests continue to boot and operate as normal...

lsb_release -rd:
Description: Ubuntu 14.04.5 LTS
Release: 14.04

CVE References

wkitty42 (wkitty42)
summary: 14.04.5 safe-upgrade from QEMU 2.0.0+dfsg-2ubuntu1.42 to QEMU
- 2.0.0+dfsg-2ubuntu1.43 kernel panics amd64 VMs
+ 2.0.0+dfsg-2ubuntu1.43 kernel panics amd64 VM guests
Revision history for this message
wkitty42 (wkitty42) wrote :

CHANGELOG: https://launchpad.net/ubuntu/+source/qemu/+changelog

2.0.0+dfsg-2ubuntu1.43
Published in trusty-updates 12 hours ago
Published in trusty-security 13 hours ago

qemu (2.0.0+dfsg-2ubuntu1.43) trusty-security; urgency=medium

  * SECURITY UPDATE: Speculative Store Bypass
    - debian/patches/CVE-2018-3639-2.patch: define the AMD 'virt-ssbd'
      CPUID feature bit in target/i386/cpu.c.
    - debian/patches/CVE-2018-3639-3.patch: define the Virt SSBD MSR and
      handling of it in target/i386/cpu.h, target/i386/kvm.c,
      target/i386/machine.c.
    - CVE-2018-3639

 -- Marc Deslauriers <email address hidden> Wed, 23 May 2018 08:03:09 -0400

Available diffs

    diff from 2.0.0+dfsg-2ubuntu1.42 (in Ubuntu) to 2.0.0+dfsg-2ubuntu1.43 (2.5 KiB)
    diff from 2.0.0+dfsg-2ubuntu1.41 to 2.0.0+dfsg-2ubuntu1.43 (3.2 KiB)

information type: Private Security → Public Security
Revision history for this message
Seth Arnold (seth-arnold) wrote :

Hello wkitty42, could you please run:

apport-collect 1776575

to automatically attach some debugging information to this bug report.

Thanks

Changed in qemu (Ubuntu):
status: New → Incomplete
Revision history for this message
wkitty42 (wkitty42) wrote :

i tried and had to install some python apport thing... it is erroring out after complaining about a bunch of things being owned by my user 1000 instead of user 0... i ran that command with sudo... so i had to clear out the apport cache credentials and rerun it without sudo... i got this (oauth token obfuscated)...

----->8 snip 8<-----

$ apport-collect 1776575

The authorization page:
 (https://launchpad.net/+authorize-token?oauth_token=xxxxxxxxxxxxxxxxxxxx&allow_permission=DESKTOP_INTEGRATION)
should be opening in your browser. Use your browser to authorize
this program to access Launchpad on your behalf.
Press any key to continue or wait (5) seconds...
Waiting to hear from Launchpad about your decision...
Traceback (most recent call last):
  File "/usr/share/apport/apport-kde", line 533, in <module>
    sys.exit(UserInterface.run_argv())
  File "/usr/lib/python2.7/dist-packages/apport/ui.py", line 658, in run_argv
    return self.run_update_report()
  File "/usr/lib/python2.7/dist-packages/apport/ui.py", line 547, in run_update_report
    self.collect_info(ignore_uninstalled=True)
  File "/usr/lib/python2.7/dist-packages/apport/ui.py", line 1012, in collect_info
    self.ui_start_info_collection_progress()
  File "/usr/share/apport/apport-kde", line 395, in ui_start_info_collection_progress
    _('The collected information can be sent to the developers '
  File "/usr/share/apport/apport-kde", line 97, in __init__
    Dialog.__init__(self, 'progress.ui', title, heading, text)
  File "/usr/share/apport/apport-kde", line 64, in __init__
    uic.loadUi(os.path.join(os.path.dirname(sys.argv[0]), ui), self)
  File "/usr/lib/python2.7/dist-packages/PyQt4/uic/__init__.py", line 238, in loadUi
    return DynamicUILoader(package).loadUi(uifile, baseinstance, resource_suffix)
  File "/usr/lib/python2.7/dist-packages/PyQt4/uic/Loader/loader.py", line 50, in __init__
    UIParser.__init__(self, QtCore, QtGui, LoaderCreatorPolicy(package))
  File "/usr/lib/python2.7/dist-packages/PyQt4/uic/uiparser.py", line 145, in __init__
    self.factory = QObjectCreator(creatorPolicy)
  File "/usr/lib/python2.7/dist-packages/PyQt4/uic/objcreator.py", line 94, in __init__
    modinfo = plugin_locals["moduleInformation"]()
  File "<string>", line 52, in moduleInformation
  File "/usr/lib/python2.7/dist-packages/qgis/__init__.py", line 26, in <module>
    from qgis.PyQt import QtCore
  File "/usr/lib/python2.7/dist-packages/qgis/PyQt/QtCore.py", line 28, in <module>
    sip.setapi(api, 2)
ValueError: API 'QDate' has already been set to version 1

----->8 snip 8<-----

i DID accept the request...

i don't know what to do, now :(

Revision history for this message
Seth Arnold (seth-arnold) wrote :

Wow, that's impressive.

Could you check dmesg for AppArmor DENIED messages, include one paragraph from /proc/cpuinfo, and look for relevant log entries in /var/log/libvirt -- please doublecheck these for passwords or other credentials, I'm not positive if libvirt's logs are paste-safe or not.

Thanks

Changed in qemu (Ubuntu):
status: Incomplete → New
Revision history for this message
Tyler Hicks (tyhicks) wrote :

Hello and thanks for the report!

Can you attach the contents of the kernel panic that you see?

Also, can you include the version number of the kernel that is running in the host and the version of the kernel running in the guest when you experience the panics?

Revision history for this message
Tyler Hicks (tyhicks) wrote :

Finally, please also attach the contents of /proc/cpuinfo in the host.

Thanks again!

tags: added: regression-update
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@wkitty42: Since the security update was for AMD processors - are you running on Intel or AMD?

Note: I marked this regression-update and we should keep and work under that urgency unless we can prove otherwise.

I spawned a set of 64 bit guests on:
  2.0.0+dfsg-2ubuntu1.42

Then I upgraded to
  2.0.0+dfsg-2ubuntu1.43

The guests still start fine.
Then I upgraded all else which e.g. meant also a related change in libvirt.
  => Still working fine.

I only had an Intel host thou, so it might be important to learn if you run on AMD hardware and if that might be a prereq to be affected? The requested /proc/cpuinfo will have all that.

Also in addition to the Kernel Panic getting the guest log could be useful in case qemu has thrown warnigns or errors.

Summarizing, please provide:
HW Info
  - cat /proc/cpuinfo
  - sudo dmidecode
Logs:
  - journalctl -xe
  - cat /var/log/libvirt/qemu/<guestname>.log
  - dmesg

Collect the logs right after the guest crashed please.

Changed in qemu (Ubuntu):
status: New → Incomplete
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Subscribing Marc who was driving the update so he is aware

Revision history for this message
wkitty42 (wkitty42) wrote :

sorry for the delay... was sleep time :)

i want to reiterate that the HOST is downgraded and running the .42 QEMU code... just to make sure we're on the same page... everything i'm reporting and looking at is on the HOST...

here's one paragraph from /proc/cpuinfo as requested...

----->8 snip 8<-----
processor : 0
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8350 Eight-Core Processor
stepping : 0
microcode : 0x6000822
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 16
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate ssbd ls_cfg_ssbd retpoline retpoline_amd npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold vmmcall bmi1
bogomips : 8036.20
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro
----->8 snip 8<-----

dmidecode is here: https://pastebin.com/LQZLmQ1g

i do not see any DENIED or AppArmour related messages in dmesg... at least not via various grep searches...

i don't see any entries in /var/log/libvirt that i can tell are relevent... i have a log for a copy of one of the 64bit VMs and the only thing that i can see that might indicate it was panicked is the "signal 15" terminations where i had to "Force Stop" the VM... that guest was created for troubleshooting... it only had the CPU section removed and the video changed to Cirrus... it uses the original guest's img... here's that log: https://pastebin.com/cs1i6s7B

for the kernel request... both were upgraded at the same time...
HOST: Linux shadow 3.13.0-151-generic #201-Ubuntu SMP Wed May 30 14:22:13 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
GUEST: Linux haven 3.13.0-151-generic #201-Ubuntu SMP Wed May 30 14:22:13 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

on the kernel panic data... all i have is two screen shots from the VM's console...
here's the 1st one with video set to VMVGA: https://imgur.com/a/esjOwJ9
here's the 2nd one with video set to Cirrus: https://imgur.com/a/JjoMSLF

i'll have to upgrade to .43 again and then capture the logs after the guest panics... i'm guessing i only need the data from the HOST??

Revision history for this message
Tyler Hicks (tyhicks) wrote :

I want to point out that the two kernel panics that were posted are from two different guest kernel versions:

  3.32.0-151-generic
  3.32.0-32-generic

That may indicate that the offending bug is in the host kernel and/or QEMU.

@wkitty42 yes, only the logs from the host are needed.

Revision history for this message
Tyler Hicks (tyhicks) wrote :

I made a typo in the kernel versions above. They should be:

  3.13.0-151-generic
  3.13.0-32-generic

Revision history for this message
wkitty42 (wkitty42) wrote : Re: [Bug 1776575] Re: 14.04.5 safe-upgrade from QEMU 2.0.0+dfsg-2ubuntu1.42 to QEMU 2.0.0+dfsg-2ubuntu1.43 kernel panics amd64 VM guests

On 06/13/2018 01:03 PM, Tyler Hicks wrote:
> I want to point out that the two kernel panics that were posted are from
> two different guest kernel versions:
>
> 3.32.0-151-generic
> 3.32.0-32-generic
>
> That may indicate that the offending bug is in the host kernel and/or
> QEMU.

that is my thinking, as well... all of my 64bit VMs panicked... that's probably
four different kernel versions on like three different distros...

> @wkitty42 yes, only the logs from the host are needed.

thanks for confirming that for me :)

Revision history for this message
Tyler Hicks (tyhicks) wrote :

@wkitty42, could you please try booting your host into an older kernel, such as 3.13.0-149.199, and using qemu 2.0.0+dfsg-2ubuntu1.43 to see if the guests still panic?

Revision history for this message
wkitty42 (wkitty42) wrote :

On 06/13/2018 01:05 PM, Tyler Hicks wrote:
> I made a typo in the kernel versions above. They should be:
>
> 3.13.0-151-generic
> 3.13.0-32-generic

FWIW: 3.13.0-32-generic might be the one where i tried booting the 10.04 ISO and
selected the "recover a broken system" option... i'm pretty sure that that's
where the second kernel panic image came from...

i also want to confirm that you guys are waiting on me to reapply the qemu .43
update files, panic my 64bit VMs again and grab those reports listed in the
following??

On 06/13/2018 02:48 AM,  Christian Ehrhardt  wrote:
 > Summarizing, please provide:
 > HW Info
 > - cat /proc/cpuinfo
 > - sudo dmidecode
 > Logs:
 > - journalctl -xe
 > - cat/var/log/libvirt/qemu/<guestname>.log
 > - dmesg
 >
 > Collect the logs right after the guest crashed please.

i don't have journalctl and don't have a clue what it is about...
do i still need the first two listed under "HW Info"??

Revision history for this message
wkitty42 (wkitty42) wrote :

On 06/13/2018 04:55 PM, Tyler Hicks wrote:
> @wkitty42, could you please try booting your host into an older kernel,
> such as 3.13.0-149.199, and using qemu 2.0.0+dfsg-2ubuntu1.43 to see if
> the guests still panic?

i can try that, yes... i just have to find a quiet time when they are not being
used too much...

it should be noted that i loath having to panic my servers for these tests...
i'm pretty sure i'm not the only admin that dislikes this part of the job ;)

Revision history for this message
Tyler Hicks (tyhicks) wrote :

I really do appreciate your testing. I'm relying on you here because while I did test this particular combination on a family 17h AMD processor and didn't see such a guest panic, I don't have a family 15h AMD processor to test on.

Revision history for this message
wkitty42 (wkitty42) wrote :
Download full text (18.1 KiB)

On 06/13/2018 02:48 AM,  Christian Ehrhardt  wrote:
> Also in addition to the Kernel Panic getting the guest log could be
> useful in case qemu has thrown warnigns or errors.

here's the new guest panic screen shot... how can i get the VM's boot log from
the time after the grub menu launches the boot of the 64bit VM till it panics?
could that contain some information that could help solve this problem?

20180615 GUEST kernel panic screen shot

> Summarizing, please provide:
> HW Info
> - cat /proc/cpuinfo

20180615 HOST AMD FX8350 8-core 4Ghz cpuinfo (1 paragraph for space):

processor : 0
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8350 Eight-Core Processor
stepping : 0
microcode : 0x6000822
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 16
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp
lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq
monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm
cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs
xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat
cpb hw_pstate ssbd ls_cfg_ssbd retpoline retpoline_amd npt lbrv svm_lock
nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold
vmmcall bmi1
bogomips : 8036.20
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

> - sudo dmidecode

a fresh one after the GUEST 64bit VM kernel panics is here:
https://pastebin.com/eB24w7ej

> Logs:
> - journalctl -xe

i don't have journalctl... it is necessary? my update log is at the end below...
that systemd thing isn't involved AFAICT...

> - cat /var/log/libvirt/qemu/<guestname>.log

$ cat /var/log/libvirt/qemu/haven-vm.log
[... normal operation with .42 ...]
2018-06-15 14:43:23.028+0000: shutting down
[... normal shutdown -h issued within guest ...]
[... update from .42 to .43 files ...]
[... NO HOST REBOOT ...]
[... restart virtual machine manager ...]
[... restart haven-vm 64bit ubuntu 14.04.5 server ...]
2018-06-15 14:58:46.362+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
QEMU_AUDIO_DRV=none /usr/bin/kvm-spice -name haven-vm -S -machine
pc-i440fx-trusty,accel=kvm,usb=off -cpu
Opteron_G5,+bmi1,+perfctr_nb,+perfctr_core,+topoext,+nodeid_msr,+tce,+lwp,+wdt,+skinit,+ibs,+osvw,+cr8legacy,+extapic,+cmp_legacy,+fxsr_opt,+mmxext,+osxsave,+monitor,+ht,+vme
-m 768 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid
8d88f791-5b85-5293-329f-943e871c4a02 -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/haven-vm.monitor,server,nowait
...

Revision history for this message
wkitty42 (wkitty42) wrote :

On 06/15/2018 11:26 AM, wkitty42 wrote:
> 20180615 GUEST kernel panic screen shot

aww, darn-it! https://imgur.com/a/He9ukKs

Revision history for this message
wkitty42 (wkitty42) wrote :

On 06/14/2018 06:47 PM, Tyler Hicks wrote:
> I really do appreciate your testing. I'm relying on you here because while I
> did test this particular combination on a family 17h AMD processor and didn't
> see such a guest panic, I don't have a family 15h AMD processor to test on.

let me know if you need anything else... i'm going to downgrade again and bring
those VMs back online... they've been down long enough and the minions are
getting quite restless...

ASIDE: BTW: i had those 13 qemu*.42 files marked as "version locked" in synaptic
but aptitude still saw the .43 ones and wanted to upgrade to them... i went
ahead and unlocked them so there wouldn't be any problems but now i'm highly
curious as to why locking them via synaptic didn't keep the .43s from showing up
with no notice that they were version locked at .42... could that have been
because of where the .42 and .43 are in the file names?

Revision history for this message
wkitty42 (wkitty42) wrote :
Download full text (16.1 KiB)

looks like my big report got cut off... sorry... here's the rest of what i had
posted... starting with the libvirt guest log just before the cut off...

On 06/13/2018 02:48 AM,  Christian Ehrhardt  wrote:>    - cat
/var/log/libvirt/qemu/<guestname>.log

$ cat /var/log/libvirt/qemu/haven-vm.log
[... normal operation with .42 ...]
2018-06-15 14:43:23.028+0000: shutting down
[... normal shutdown -h issued within guest ...]
[... update from .42 to .43 files ...]
[... NO HOST REBOOT ...]
[... restart virtual machine manager ...]
[... restart haven-vm 64bit ubuntu 14.04.5 server ...]
2018-06-15 14:58:46.362+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
QEMU_AUDIO_DRV=none /usr/bin/kvm-spice -name haven-vm -S -machine
pc-i440fx-trusty,accel=kvm,usb=off -cpu
Opteron_G5,+bmi1,+perfctr_nb,+perfctr_core,+topoext,+nodeid_msr,+tce,+lwp,+wdt,+skinit,+ibs,+osvw,+cr8legacy,+extapic,+cmp_legacy,+fxsr_opt,+mmxext,+osxsave,+monitor,+ht,+vme
-m 768 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid
8d88f791-5b85-5293-329f-943e871c4a02 -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/haven-vm.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/libvirt/images/haven-vm.img,if=none,id=drive-virtio-disk0,format=raw
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev
tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6c:88:6a,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
-vnc 127.0.0.1:0 -device vmware-svga,id=video0,bus=pci.0,addr=0x2 -device
intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
char device redirected to /dev/pts/2 (label charserial0)
<eof>

 > - dmesg

$ dmesg | tail -n 50
[189401.694498] RED: port 4(vnet2) entered disabled state
[189401.695136] device vnet2 left promiscuous mode
[189401.695146] RED: port 4(vnet2) entered disabled state
[189402.022302] type=1400 audit(1529007011.473:117): apparmor="STATUS"
operation="profile_remove" profile="unconfined"
name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=29425 comm="apparmor_parser"
[189412.938869] type=1400 audit(1529007022.390:118): apparmor="STATUS"
operation="profile_load" profile="unconfined"
name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=29440 comm="apparmor_parser"
[189413.032182] device vnet2 entered promiscuous mode
[189413.064318] RED: port 4(vnet2) entered forwarding state
[189413.064369] RED: port 4(vnet2) entered forwarding state
[189413.524620] kvm: zapping shadow pages for mmio generation wraparound
[190751.089128] RED: port 4(vnet2) entered disabled state
[190751.089351] device vnet2 left promiscuous mode
[190751.089357] RED: port 4(...

Revision history for this message
wkitty42 (wkitty42) wrote :
Download full text (4.4 KiB)

good grief... and the rest... WTH?

> - dmesg

$ dmesg | tail -n 50
[189401.694498] RED: port 4(vnet2) entered disabled state
[189401.695136] device vnet2 left promiscuous mode
[189401.695146] RED: port 4(vnet2) entered disabled state
[189402.022302] type=1400 audit(1529007011.473:117): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=29425 comm="apparmor_parser"
[189412.938869] type=1400 audit(1529007022.390:118): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=29440 comm="apparmor_parser"
[189413.032182] device vnet2 entered promiscuous mode
[189413.064318] RED: port 4(vnet2) entered forwarding state
[189413.064369] RED: port 4(vnet2) entered forwarding state
[189413.524620] kvm: zapping shadow pages for mmio generation wraparound
[190751.089128] RED: port 4(vnet2) entered disabled state
[190751.089351] device vnet2 left promiscuous mode
[190751.089357] RED: port 4(vnet2) entered disabled state
[190756.389518] type=1400 audit(1529008365.840:119): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=31466 comm="apparmor_parser"
[190784.008591] type=1400 audit(1529008393.460:120): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=31505 comm="apparmor_parser"
[190784.104215] device vnet2 entered promiscuous mode
[190784.120319] RED: port 4(vnet2) entered forwarding state
[190784.120341] RED: port 4(vnet2) entered forwarding state
[190788.883915] kvm: zapping shadow pages for mmio generation wraparound
[192525.623898] RED: port 4(vnet2) entered disabled state
[192525.624178] device vnet2 left promiscuous mode
[192525.624217] RED: port 4(vnet2) entered disabled state
[192526.174179] type=1400 audit(1529010135.625:121): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=1282 comm="apparmor_parser"
[192594.838386] type=1400 audit(1529010204.289:122): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=1377 comm="apparmor_parser"
[192594.928167] device vnet2 entered promiscuous mode
[192594.960225] RED: port 4(vnet2) entered forwarding state
[192594.960234] RED: port 4(vnet2) entered forwarding state
[192598.131548] kvm: zapping shadow pages for mmio generation wraparound
[207463.151824] RED: port 4(vnet2) entered disabled state
[207463.151976] device vnet2 left promiscuous mode
[207463.151989] RED: port 4(vnet2) entered disabled state
[207463.443798] type=1400 audit(1529025072.894:123): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-30c5afb2-602a-1c9c-c940-0b47cadbe52c" pid=20089 comm="apparmor_parser"
[217591.715997] sqpack[32402]: segfault at 1 ip 00007f66001cb943 sp 00007ffda3b61c00 error 4 in libc-2.19.so[7f6600180000+1be000]
[228402.112955] hrtimer: interrupt took 41067 ns
[256193.281642] RED: port 3(vnet1) entered disabled state
[256193.282064] device vnet1 left promiscuous mode
[256193.282...

Read more...

Revision history for this message
wkitty42 (wkitty42) wrote :
Download full text (9.6 KiB)

and the last...

20180615 HOST update log QEMU*.42 -> QEMU*.43:

$ cat dosysupdate.log
Execute "sudo aptitude -V safe-upgrade"? Yes/no
Response: Yes. Executing "sudo aptitude -V safe-upgrade"...

Reading package lists...
Building dependency tree...
Reading state information...
Reading extended state information...
Initializing package states...
The following packages will be upgraded:
  debootstrap [1.0.59ubuntu0.9 -> 1.0.59ubuntu0.10]
  file [1:5.14-2ubuntu3.3 -> 1:5.14-2ubuntu3.4]
  gnupg-agent [2.0.22-3ubuntu1.3 -> 2.0.22-3ubuntu1.4]
  gnupg2 [2.0.22-3ubuntu1.3 -> 2.0.22-3ubuntu1.4]
  gpgsm [2.0.22-3ubuntu1.3 -> 2.0.22-3ubuntu1.4]
  libmagic1 [1:5.14-2ubuntu3.3 -> 1:5.14-2ubuntu3.4]
  libperl5.18 [5.18.2-2ubuntu1.4 -> 5.18.2-2ubuntu1.6]
  libruby1.9.1 [1.9.3.484-2ubuntu1.11 -> 1.9.3.484-2ubuntu1.12]
  perl [5.18.2-2ubuntu1.4 -> 5.18.2-2ubuntu1.6]
  perl-base [5.18.2-2ubuntu1.4 -> 5.18.2-2ubuntu1.6]
  perl-modules [5.18.2-2ubuntu1.4 -> 5.18.2-2ubuntu1.6]
  python3-magic [1:5.14-2ubuntu3.3 -> 1:5.14-2ubuntu3.4]
  qemu [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-keymaps [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-kvm [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-arm [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-common [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-mips [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-misc [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-ppc [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-sparc [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-system-x86 [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-user [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  qemu-utils [2.0.0+dfsg-2ubuntu1.42 -> 2.0.0+dfsg-2ubuntu1.43]
  ruby1.9.1 [1.9.3.484-2ubuntu1.11 -> 1.9.3.484-2ubuntu1.12]
  scdaemon [2.0.22-3ubuntu1.3 -> 2.0.22-3ubuntu1.4]
27 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.7 MB/32.6 MB of archives. After unpacking 13.3 kB will be used.
Do you want to continue? [Y/n/?] Writing extended state information...
Get: 1 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main perl amd64 5.18.2-2ubuntu1.6 [2,634 kB]
Get: 2 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main libperl5.18 amd64 5.18.2-2ubuntu1.6 [1,322 B]
Get: 3 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main perl-base amd64 5.18.2-2ubuntu1.6 [1,152 kB]
Get: 4 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main perl-modules all 5.18.2-2ubuntu1.6 [2,684 kB]
Get: 5 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main file amd64 1:5.14-2ubuntu3.4 [19.4 kB]
Get: 6 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main libmagic1 amd64 1:5.14-2ubuntu3.4 [185 kB]
Get: 7 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main gnupg2 amd64 2.0.22-3ubuntu1.4 [680 kB]
Get: 8 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main gnupg-agent amd64 2.0.22-3ubuntu1.4 [231 kB]
Get: 9 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main gpgsm amd64 2.0.22-3ubuntu1.4 [192 kB]
Get: 10 h...

Read more...

Revision history for this message
wkitty42 (wkitty42) wrote :

done... what is missing?

Changed in qemu (Ubuntu):
status: Incomplete → Confirmed
Simon Quigley (tsimonq2)
information type: Public Security → Public
information type: Public → Public Security
Revision history for this message
wkitty42 (wkitty42) wrote :

@tyhicks: i totally forgot you asked me to try testing with the HOST running an older kernel... my amd64 GUEST VMs are NOT kernel panicking :shock: :shock: :shock:

20180616 HOST uname -a:
Linux shadow 3.13.0-149-generic #199-Ubuntu SMP Thu May 17 10:12:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

20180616 HOST cpuinfo:
same as previous posted

20180616 HOST dmidecode:
suspect is the same but here is the latest one, https://pastebin.com/VW3x9NaP

20180616 HOST cat /var/log/libvirt/qemu/haven-vm.log:
2018-06-16 14:13:39.077+0000: shutting down
[... upgrade to qemu*.43 ...]
[... HOST reboot ...]
2018-06-16 14:17:50.050+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm-spice -name haven-vm -S -machine pc-i440fx-trusty,accel=kvm,usb=off -cpu Opteron_G5,+bmi1,+perfctr_nb,+perfctr_core,+topoext,+nodeid_msr,+tce,+lwp,+wdt,+skinit,+ibs,+osvw,+cr8legacy,+extapic,+cmp_legacy,+fxsr_opt,+mmxext,+osxsave,+monitor,+ht,+vme -m 768 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 8d88f791-5b85-5293-329f-943e871c4a02 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/haven-vm.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/haven-vm.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6c:88:6a,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:1 -device vmware-svga,id=video0,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
char device redirected to /dev/pts/7 (label charserial0)

20180616 HOST dmesg (full):
https://pastebin.com/demTJw2V

20180616 HOST update log QEMU*.42 -> QEMU*.43:
https://pastebin.com/8EfZu7Av

Revision history for this message
wkitty42 (wkitty42) wrote :

FWIW: here's the info for the GUEST amd64 VM that has been panicking...

20180616 GUEST uname -a:
Linux haven 3.13.0-151-generic #201-Ubuntu SMP Wed May 30 14:22:13 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

20180616 GUEST cpuinfo:
processor : 0
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD Opteron 63xx class CPU
stepping : 0
microcode : 0x1000065
cpu MHz : 4018.284
cache size : 512 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb lm rep_good nopl extd_apicid pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic popcnt aes xsave avx f16c hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw xop fma4 tbm arat retpoline retpoline_amd vmmcall bmi1
bogomips : 8036.56
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

20180616 GUEST dmidecode:
https://pastebin.com/bgEvr6LK

20180616 GUEST dmesg (full):
https://pastebin.com/fBZs7Mit

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.