Comment 2 for bug 1643911

Revision history for this message
Lee Yarwood (lyarwood) wrote :

Just saw this on stable/newton and having looked at the logs I think this is a logging issue in libvirtd when terminating an instance created by the ImagesTestJSON tests.

- Attempt to terminate the ImagesTestJSON instance prior to libvirtd dying :

http://logs.openstack.org/61/407961/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/3dc2b54/logs/screen-n-cpu.txt.gz#_2016-12-12_07_48_37_547

- Call to qemuDomainLogContextRead that I think is triggering this issue :

http://logs.openstack.org/61/407961/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/3dc2b54/logs/libvirt/libvirtd.txt.gz#_2016-12-12_07_48_37_835

- Further call logged to virLogProbablyLogMessage in syslog :

http://logs.openstack.org/61/407961/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/3dc2b54/logs/syslog.txt.gz#_Dec_12_07_48_37

Kashyap, could you raise this with the libvirt folks for their take on this?

~~~
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: *** Error in `/usr/sbin/libvirtd': malloc(): memory corruption: 0x000055a98c7c6810 ***
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: ======= Backtrace: =========
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fc916b557e5]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(+0x8181e)[0x7fc916b5f81e]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fc916b615d4]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(realloc+0x358)[0x7fc916b61e68]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(+0xe53a2)[0x7fc916bc33a2]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(regexec+0xb3)[0x7fc916bc59c3]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/lib/x86_64-linux-gnu/libvirt.so.0(virLogProbablyLogMessage+0x1f)[0x7fc917563a5f]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so(+0x7d080)[0x7fc8c7ddf080]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so(+0x933dc)[0x7fc8c7df53dc]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/lib/x86_64-linux-gnu/libvirt.so.0(virEventPollRunOnce+0x7b3)[0x7fc91754d393]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/lib/x86_64-linux-gnu/libvirt.so.0(virEventRunDefaultImpl+0x41)[0x7fc91754bc01]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/lib/x86_64-linux-gnu/libvirt.so.0(virNetDaemonRun+0x14d)[0x7fc917697b4d]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/sbin/libvirtd(+0x14407)[0x55a98c480407]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fc916afe830]
Dec 12 07:48:37 ubuntu-xenial-osic-cloud1-disk-6107639 libvirtd[29343]: /usr/sbin/libvirtd(+0x14bd9)[0x55a98c480bd9]
~~~