Logging CPU incompatibility when attempting live migration fails

[Darren Worrall]

Running nova --no-cache live-migration --block-migrate myvm myhost returns:

ERROR: Live migration of instance <uuid> to host myhost failed (HTTP 400) (Request-ID: req-<uuid>)

There's a stacktrace in the nova-compute.log on the target host:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 275, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in wrapped
    temp_level, payload)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in wrapped
    return f(*args, **kw)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2076, in check_can_live_migrate_destination
    instance, block_migration, disk_over_commit)
  File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2238, in check_can_live_migrate_destination
    self._compare_cpu(source_cpu_info)
  File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2366, in _compare_cpu
    LOG.error(reason=m % locals())
TypeError: error() takes at least 2 arguments (1 given)

LOG.error doesnt take reason, simply removing the reason= bubbles up the correct exception:

InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.

[Darren Worrall (dazworrall)]

nova-compute:
  Installed: 2012.2-0ubuntu5~cloud0
  Candidate: 2012.2-0ubuntu5~cloud0
  Version table:
 *** 2012.2-0ubuntu5~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ precise-updates/folsom/main amd64 Packages
        100 /var/lib/dpkg/status
     2012.1.3+stable-20120827-4d2a4afe-0ubuntu1 0
        500 http://gb.archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
     2012.1+stable~20120612-3ee026e-0ubuntu1.3 0
        500 http://security.ubuntu.com/ubuntu/ precise-security/main amd64 Packages
     2012.1-0ubuntu2 0
        500 http://gb.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages

[Darren Worrall (dazworrall)]

Tiny pull request: https://github.com/openstack/nova/pull/17

[Davanum Srinivas (DIMS) (dims-v)]

Darren,

Please use the workflow here - http://wiki.openstack.org/GerritWorkflow to submit changes.

thanks,
dims

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/15741

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/15741
Committed: http://github.com/openstack/nova/commit/c63880cc78ab5e0d88457cf4cac20d3d55421ba0
Submitter: Jenkins
Branch: master

commit c63880cc78ab5e0d88457cf4cac20d3d55421ba0
Author: Darren Worrall <email address hidden>
Date: Thu Nov 8 10:36:22 2012 +0000

    Fix incorrect LOG.error usage in _compare_cpu

    bug 1076308

    Change-Id: I30a6447f005e708107dc01578664a4cb2bb14439

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/16046

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/16046
Committed: http://github.com/openstack/nova/commit/92eddd2d471269835ade19412a28539fa9246bfc
Submitter: Jenkins
Branch: stable/folsom

commit 92eddd2d471269835ade19412a28539fa9246bfc
Author: Darren Worrall <email address hidden>
Date: Thu Nov 8 10:36:22 2012 +0000

    Fix incorrect LOG.error usage in _compare_cpu

    bug 1076308

    Change-Id: I30a6447f005e708107dc01578664a4cb2bb14439

[Dražen Lučanin (kermit666)]

So, what is the solution to "InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.", then? I am aware that I am migrating between Intel and AMD machines, but isn't there a way to make this work?

[maying (breeze7086)]

I find when do the migration,nova detect that two host have different CPU
flags(cat /proc/cpuinfo),and it will rise "InvalidCPUInfo: Unacceptable CPU
info: CPU doesn't have compatibility"
If you want to solve this problem, better to choose hosts within same CPU
or try to change the code in this section.

2012/12/13 Dražen Lučanin <email address hidden>

> So, what is the solution to "InvalidCPUInfo: Unacceptable CPU info: CPU
> doesn't have compatibility.", then? I am aware that I am migrating
> between Intel and AMD machines, but isn't there a way to make this work?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1076308
>
> Title:
> Logging CPU incompatibility when attempting live migration fails
>
> Status in OpenStack Compute (Nova):
> Fix Released
> Status in OpenStack Compute (nova) folsom series:
> Fix Released
> Status in “nova” package in Ubuntu:
> Fix Released
> Status in “nova” source package in Quantal:
> Confirmed
>
> Bug description:
> Running nova --no-cache live-migration --block-migrate myvm myhost
> returns:
>
> ERROR: Live migration of instance <uuid> to host myhost failed (HTTP
> 400) (Request-ID: req-<uuid>)
>
> There's a stacktrace in the nova-compute.log on the target host:
>
> Traceback (most recent call last):
> File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
> 275, in _process_data
> rval = self.proxy.dispatch(ctxt, version, method, **args)
> File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
> line 145, in dispatch
> return getattr(proxyobj, method)(ctxt, **kwargs)
> File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117,
> in wrapped
> temp_level, payload)
> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> self.gen.next()
> File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in
> wrapped
> return f(*args, **kw)
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 2076, in check_can_live_migrate_destination
> instance, block_migration, disk_over_commit)
> File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
> line 2238, in check_can_live_migrate_destination
> self._compare_cpu(source_cpu_info)
> File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
> line 2366, in _compare_cpu
> LOG.error(reason=m % locals())
> TypeError: error() takes at least 2 arguments (1 given)
>
> LOG.error doesnt take reason, simply removing the reason= bubbles up
> the correct exception:
>
> InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/1076308/+subscriptions
>

[Clint Byrum (clint-fewbar)]

Hello Darren, or anyone else affected,

Accepted nova into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/2012.2.1+stable-20121212-a99a802e-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package nova - 2012.2.1+stable-20121212-a99a802e-0ubuntu1

---------------
nova (2012.2.1+stable-20121212-a99a802e-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - debian/control: Ensure novaclient is upgraded with nova,
      require python-keystoneclient >= 1:2.9.0. (LP: #1073289)
    - d/p/avoid_setuptools_git_dependency.patch: Refresh.
  * Dropped patches, applied upstream:
    - debian/patches/CVE-2012-5625.patch: [a99a802]
  * Resynchronize with stable/folsom (b55014ca) (LP: #1085255):
    - [a99a802] create_lvm_image allocates dirty blocks (LP: #1070539)
    - [670b388] RPC exchange name defaults to 'openstack' (LP: #1083944)
    - [3ede373] disassociate_floating_ip with multi_host=True fails
      (LP: #1074437)
    - [22d7c3b] libvirt imagecache should handle shared image storage
      (LP: #1075018)
    - [e787786] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [9265eb0] live_migration missing migrate_data parameter in Hyper-V driver
      (LP: #1066513)
    - [3d99848] use_single_default_gateway does not function correctly
      (LP: #1075859)
    - [65a2d0a] resize does not migrate DHCP host information (LP: #1065440)
    - [102c76b] Nova backup image fails (LP: #1065053)
    - [48a3521] Fix config-file overrides for nova-dhcpbridge
    - [69663ee] Cloudpipe in Folsom: no such option: cnt_vpn_clients
      (LP: #1069573)
    - [6e47cc8] DisassociateAddress can cause Internal Server Error
      (LP: #1080406)
    - [22c3d7b] API calls to dis-associate an auto-assigned floating IP should
      return proper warning (LP: #1061499)
    - [bd11d15] libvirt: if exception raised during volume_detach, volume state
      is inconsistent (LP: #1057756)
    - [dcb59c3] admin can't describe all images in ec2 api (LP: #1070138)
    - [78de622] Incorrect Exception raised during Create server when metadata
      over 255 characters (LP: #1004007)
    - [c313de4] Fixed IP isn't released before updating DHCP host file
      (LP: #1078718)
    - [f4ab42d] Enabling Return Reservation ID with XML create server request
      returns no body (LP: #1061124)
    - [3db2a38] 'BackupCreate' should accept rotation parameter greater than or
      equal to zero (LP: #1071168)
    - [f7e5dde] libvirt reboot sometimes fails to reattach volumes
      (LP: #1073720)
    - [ff776d4] libvirt: detaching volume may fail while terminating other
      instances on the same host concurrently (LP: #1060836)
    - [85a8bc2] Used instance uuid rather than id in remove-fixed-ip
    - [42a85c0] Fix error on invalid delete_on_termination value
    - [6a17579] xenapi migrations fail w/ swap (LP: #1064083)
    - [97649b8] attach-time field for volumes is not updated for detach volume
      (LP: #1056122)
    - [8f6a718] libvirt: rebuild is not using kernel and ramdisk associated with
      the new image (LP: #1060925)
    - [fbe835f] live-migration and volume host assignement (LP: #1066887)
    - [c2a9150] typo prevents volume_tmp_dir flag from working (LP: #1071536)
    - [93efa21] Instances deleted during spawn leak network allocations
      (LP: #1068716)
    - [ebabd02] After restartin...