IncompatibleObjectVersion: Version 1.2 of PciDeviceList is not supported

[yanchen]

OS : Ubuntu14.04 Server
Openstack Version: kilo

when I upgrade the service nova-compute pkg from 2015.1.0-0ubuntu1.1~cloud0 to 2015.1.1-0ubuntu1~cloud2. The problem was recognized.

nova-compute service log is :

2015-08-31 09:20:30.248 45172 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-08-31 09:20:30.305 45172 INFO oslo_messaging._drivers.impl_rabbit [req-9d361b32-76bf-41c6-a097-beba82a6c2bd - - - - -] Connecting to AMQP server on 10.11.101.3:5672
2015-08-31 09:20:30.322 45172 INFO oslo_messaging._drivers.impl_rabbit [req-9d361b32-76bf-41c6-a097-beba82a6c2bd - - - - -] Connected to AMQP server on 10.11.101.3:5672
2015-08-31 09:20:30.330 45172 INFO oslo_messaging._drivers.impl_rabbit [req-9d361b32-76bf-41c6-a097-beba82a6c2bd - - - - -] Connecting to AMQP server on 10.11.101.3:5672
2015-08-31 09:20:30.349 45172 INFO oslo_messaging._drivers.impl_rabbit [req-9d361b32-76bf-41c6-a097-beba82a6c2bd - - - - -] Connected to AMQP server on 10.11.101.3:5672
2015-08-31 09:20:30.388 45172 INFO nova.service [-] Starting compute node (version 2015.1.1)
2015-08-31 09:20:30.405 45172 INFO nova.virt.libvirt.driver [-] Connection event '1' reason 'None'
2015-08-31 09:20:30.441 45172 INFO nova.virt.libvirt.host [req-43070318-b20b-4d24-b079-815fab4c1ea7 - - - - -] Libvirt host capabilities <capabilities>
......
......
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup timeout=timeout, retry=retry)
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup retry=retry)
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup raise result
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup IncompatibleObjectVersion_Remote: Version 1.2 of PciDeviceList is not supported
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup executor_callback))
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup executor_callback)
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup result = func(ctxt, **new_args)
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 430, in object_class_action
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup objver)
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 324, in obj_class_from_name
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup supported=latest_ver)
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup IncompatibleObjectVersion: Version 1.2 of PciDeviceList is not supported
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup
2015-08-31 09:20:31.022 45172 TRACE nova.openstack.common.threadgroup

nova-compute pkg detail version is :
root@host214:~# dpkg -l | grep nova
ii nova-common 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - common files
ii nova-compute 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node base
ii nova-compute-kvm 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node libvirt support
ii python-nova 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute Python libraries
ii python-novaclient 1:2.22.0-0ubuntu1~cloud0 all client library for OpenStack Compute API

libvirt pkg version is :
root@host214:~# dpkg -l | grep libvirt
ii libvirt-bin 1.2.12-0ubuntu14.1~cloud0 amd64 programs for the libvirt library
ii libvirt0 1.2.12-0ubuntu14.1~cloud0 amd64 library for interfacing with different virtualization systems
ii nova-compute-libvirt 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.2.2-0ubuntu2 amd64 libvirt Python bindings

and qemu pkg version is :
root@host214:~# dpkg -l | grep qemu
ii ipxe-qemu 1.0.0+git-20131111.c3d1e78-2ubuntu1.1 all PXE boot firmware - ROM images for qemu
ii qemu-slof 20140630+dfsg-1ubuntu1~14.04 all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries
ii qemu-system-arm 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-mips 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (miscelaneous)
ii qemu-system-ppc 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:2.2+dfsg-5expubuntu9.3~cloud0 amd64 QEMU utilities

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Samir Ibradžić (sibradzic)]

I've run into this yesterday, right after nova-compute upgrade to 2015.1.1 on all compute nodes in my environment. Turned out that openstack-nova 2015.1.0 (scheduler or perhaps one of its dependencies?) on controller is not something that nova-compute 2015.1.1 on compute nodes can live with!
Upgrading openstack-nova* and all of its dependencies on a controller to 2015.1.1 fixed the thing.

[Tobias Urdin (tobias-urdin)]

I had the same issue after upgrading nova components.
The issue was that nova-conductor and nova-compute services did not use the same version.

I upgraded using apt-get upgrade but it didn't seem to update nova-conductor. After a apt-get -f upgrade nova-conductor the nova-conductor package was upgraded to same version as nova-compute and then it worked. Confirm by checking the package versions (dpkg -l | grep nova).

[Xabier (xelkano)]

I am facing exactly the same issue after upgrading from JUNO to KILO, Is there any work around ?