OpenStack Compute (nova)

live-migration --block-migrate <UUID> fails: XML error: CPU feature `pdpe1gb' specified more than once

Bug #1322681 reported by James Page
44
This bug affects 9 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
New
Undecided
Unassigned
nova (Fedora)
New
Undecided
Unassigned
nova (Ubuntu)
Triaged
Medium
Unassigned

Bug Description

2014-05-23 15:32:58.767 57916 INFO nova.compute.resource_tracker [-] Compute_service record updated for chakora:chakora.1ss.qa.lexington
2014-05-23 15:33:42.551 57916 INFO nova.virt.libvirt.driver [req-9d8c6bac-74ea-4876-9fb2-729f2f525fb2 ff364dfc08b84aab871a802888cfbd76 ce780a90138b466f8a0343d487e3198c] Instance launched has CPU info:
{"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64", "features": ["pdpe1gb", "osxsave", "dca", "pcid", "pdcm", "xtpr", "tm2", "est", "smx", "vmx", "ds_cpl", "monitor", "dtes64", "pbe", "tm", "ht", "ss", "acpi", "ds", "vme", "pdpe1gb", "osxsave", "dca", "pcid", "pdcm", "xtpr", "tm2", "est", "smx", "vmx", "ds_cpl", "monitor", "dtes64", "pbe", "tm", "ht", "ss", "acpi", "ds", "vme", "lahf_lm", "lm", "rdtscp", "nx", "syscall", "avx", "xsave", "aes", "tsc-deadline", "popcnt", "x2apic", "sse4.2", "sse4.1", "cx16", "ssse3", "pclmuldq", "pni", "sse2", "sse", "fxsr", "mmx", "clflush", "pse36", "pat", "cmov", "mca", "pge", "mtrr", "sep", "apic", "cx8", "mce", "pae", "msr", "tsc", "pse", "de", "fpu"], "topology": {"cores": 8, "threads": 2, "sockets": 1}}
2014-05-23 15:33:42.556 57916 ERROR nova.virt.libvirt.driver [req-9d8c6bac-74ea-4876-9fb2-729f2f525fb2 ff364dfc08b84aab871a802888cfbd76 ce780a90138b466f8a0343d487e3198c] CPU doesn't have compatibility.

XML error: CPU feature `pdpe1gb' specified more than once

Refer to http://libvirt.org/html/libvirt-libvirt.html#virCPUCompareResult
2014-05-23 15:33:42.568 57916 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: XML error: CPU feature `pdpe1gb' specified more than once
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher payload)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info())
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4418, in check_can_live_migrate_destination
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher block_migration, disk_over_commit)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4189, in check_can_live_migrate_destination
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher self._compare_cpu(source_cpu_info)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4331, in _compare_cpu
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher LOG.error(m, {'ret': ret, 'u': u})
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4327, in _compare_cpu
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher ret = self._conn.compareCPU(cpu.to_xml(), 0)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher rv = execute(f,*args,**kwargs)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher rv = meth(*args,**kwargs)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3191, in compareCPU
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virConnectCompareCPU() failed', conn=self)
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher libvirtError: XML error: CPU feature `pdpe1gb' specified more than once
2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher
2014-05-23 15:33:42.569 57916 ERROR oslo.messaging._drivers.common [-] Returning exception XML error: CPU feature `pdpe1gb' specified more than once to caller
2014-05-23 15:33:42.570 57916 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped\n payload)\n', ' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function\n e, sys.exc_info())\n', ' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4418, in check_can_live_migrate_destination\n block_migration, disk_over_commit)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4189, in check_can_live_migrate_destination\n self._compare_cpu(source_cpu_info)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4331, in _compare_cpu\n LOG.error(m, {\'ret\': ret, \'u\': u})\n', ' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4327, in _compare_cpu\n ret = self._conn.compareCPU(cpu.to_xml(), 0)\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call\n rv = execute(f,*args,**kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker\n rv = meth(*args,**kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3191, in compareCPU\n if ret == -1: raise libvirtError (\'virConnectCompareCPU() failed\', conn=self)\n', "libvirtError: XML error: CPU feature `pdpe1gb' specified more than once\n"]
2014-05-23 15:33:58.940 57916 AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: nova-compute 1:2014.1-0ubuntu1
ProcVersionSignature: User Name 3.13.0-24.46-generic 3.13.9
Uname: Linux 3.13.0-24-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.2
Architecture: amd64
Date: Fri May 23 15:43:40 2014
NovaConf: Error: [Errno 13] Permission denied: '/etc/nova/nova.conf'
PackageArchitecture: all
ProcEnviron:
 TERM=screen-bce
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: nova
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.nova.rootwrap.d.compute.filters: [deleted]

Revision history for this message
James Page (james-page) wrote :
Revision history for this message
James Page (james-page) wrote :

The above error log is from the target host for live-migration

James Page (james-page)
summary: - live-migration --block-migrate <UUID> fails
+ live-migration --block-migrate <UUID> fails: XML error: CPU feature
+ `pdpe1gb' specified more than once
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in nova (Ubuntu):
status: New → Confirmed
Revision history for this message
Botho Hohbaum (dynamitexxl) wrote :

i had the same problem with icehouse on ubuntu 14.04 LTS and fixed it with the following change in
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py

at line 4338 ff i removed

        for f in info['features']:
            cpu.add_feature(vconfig.LibvirtConfigCPUFeature(f))

and replaced it with

        test = []
        for f in info['features']:
            if f not in test:
                test.append(f)
                cpu.add_feature(vconfig.LibvirtConfigCPUFeature(f))

maybe this helps someone until the fix is released.

Revision history for this message
James Page (james-page) wrote :

I think this is related to bug 1267191 which is currently in verfication in trusty-proposed.

Changed in nova (Ubuntu):
importance: Undecided → Medium
status: Confirmed → Triaged
Revision history for this message
Prateek khushalani (prateek-khushalani) wrote : Re: [Bug 1322681] Re: live-migration --block-migrate <UUID> fails: XML error: CPU feature `pdpe1gb' specified more than once
Download full text (12.1 KiB)

Hi,

its a bug it the code. The workaround is to comment the function in the
libvirt code which checks the flags of cpu while migration. Do make sure
manually that the cpu flags of both the compute nodes are same.

Regards,
Prateek Khushalani
On Aug 1, 2014 6:51 PM, "James Page" <email address hidden> wrote:

> *** This bug is a duplicate of bug 1267191 ***
> https://bugs.launchpad.net/bugs/1267191
>
> I think this is related to bug 1267191 which is currently in verfication
> in trusty-proposed.
>
> ** Changed in: nova (Ubuntu)
> Importance: Undecided => Medium
>
> ** Changed in: nova (Ubuntu)
> Status: Confirmed => Triaged
>
> ** This bug has been marked a duplicate of bug 1267191
> openstack-nova-compute service fails with - libvirtError: internal
> error: CPU feature `avx' specified more than once
>
> --
> You received this bug notification because you are subscribed to
> OpenStack.
> Matching subscriptions: Prateek Khushalani
> https://bugs.launchpad.net/bugs/1322681
>
> Title:
> live-migration --block-migrate <UUID> fails: XML error: CPU feature
> `pdpe1gb' specified more than once
>
> Status in OpenStack Compute (Nova):
> New
> Status in “nova” package in Ubuntu:
> Triaged
> Status in “nova” package in Fedora:
> New
>
> Bug description:
> 2014-05-23 15:32:58.767 57916 INFO nova.compute.resource_tracker [-]
> Compute_service record updated for chakora:chakora.1ss.qa.lexington
> 2014-05-23 15:33:42.551 57916 INFO nova.virt.libvirt.driver
> [req-9d8c6bac-74ea-4876-9fb2-729f2f525fb2 ff364dfc08b84aab871a802888cfbd76
> ce780a90138b466f8a0343d487e3198c] Instance launched has CPU info:
> {"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64",
> "features": ["pdpe1gb", "osxsave", "dca", "pcid", "pdcm", "xtpr", "tm2",
> "est", "smx", "vmx", "ds_cpl", "monitor", "dtes64", "pbe", "tm", "ht",
> "ss", "acpi", "ds", "vme", "pdpe1gb", "osxsave", "dca", "pcid", "pdcm",
> "xtpr", "tm2", "est", "smx", "vmx", "ds_cpl", "monitor", "dtes64", "pbe",
> "tm", "ht", "ss", "acpi", "ds", "vme", "lahf_lm", "lm", "rdtscp", "nx",
> "syscall", "avx", "xsave", "aes", "tsc-deadline", "popcnt", "x2apic",
> "sse4.2", "sse4.1", "cx16", "ssse3", "pclmuldq", "pni", "sse2", "sse",
> "fxsr", "mmx", "clflush", "pse36", "pat", "cmov", "mca", "pge", "mtrr",
> "sep", "apic", "cx8", "mce", "pae", "msr", "tsc", "pse", "de", "fpu"],
> "topology": {"cores": 8, "threads": 2, "sockets": 1}}
> 2014-05-23 15:33:42.556 57916 ERROR nova.virt.libvirt.driver
> [req-9d8c6bac-74ea-4876-9fb2-729f2f525fb2 ff364dfc08b84aab871a802888cfbd76
> ce780a90138b466f8a0343d487e3198c] CPU doesn't have compatibility.
>
> XML error: CPU feature `pdpe1gb' specified more than once
>
> Refer to
> http://libvirt.org/html/libvirt-libvirt.html#virCPUCompareResult
> 2014-05-23 15:33:42.568 57916 ERROR oslo.messaging.rpc.dispatcher [-]
> Exception during message handling: XML error: CPU feature `pdpe1gb'
> specified more than once
> 2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher
> Traceback (most recent call last):
> 2014-05-23 15:33:42.568 57916 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/disp...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.