live migration failed

Bug #1302334 reported by Guangya Liu (Jay Lau)
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Dan Smith

Bug Description

When I try to live migration a VM, I got the following exception from source host.

2014-04-04 12:50:20.330 8862 INFO nova.compute.manager [-] [instance: 160fb719-7f84-466a-a19d-9284dd6d56fa] NV-FA2EA85 _post_live_migration() is started..
2014-04-04 12:50:20.371 8862 ERROR nova.openstack.common.loopingcall [-] in fixed duration looping call
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall Traceback (most recent call last):
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/openstack/common/loopingcall.py", line 78, in _inner
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall self.f(*self.args, **self.kw)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 4495, in wait_for_live_migration
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall migrate_data)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/exception.py", line 88, in wrapped
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall payload)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall six.reraise(self.type_, self.value, self.tb)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/exception.py", line 71, in wrapped
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall return f(self, context, *args, **kw)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 315, in decorated_function
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall e, sys.exc_info())
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall six.reraise(self.type_, self.value, self.tb)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 302, in decorated_function
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall return function(self, context, *args, **kwargs)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 4600, in _post_live_migration
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall ctxt, instance.uuid)
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall AttributeError: 'dict' object has no attribute 'uuid'
2014-04-04 12:50:20.371 8862 TRACE nova.openstack.common.loopingcall

Changed in nova:
assignee: nobody → Jay Lau (jay-lau-513)
importance: Undecided → High
Revision history for this message
Guangya Liu (Jay Lau) (jay-lau-513) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/85244

Changed in nova:
status: New → In Progress
tags: added: icehouse-rc-potential
Changed in nova:
milestone: none → icehouse-rc2
Revision history for this message
Matt Riedemann (mriedem) wrote :

Alternative fix (revert of the original patch) is here: https://review.openstack.org/#/c/85394/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/85421

Changed in nova:
assignee: Jay Lau (jay-lau-513) → nobody
assignee: nobody → Dan Smith (danms)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/85394
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=fb4c7679846c0e7a1264816c8ce461c70725cf3d
Submitter: Jenkins
Branch: master

commit fb4c7679846c0e7a1264816c8ce461c70725cf3d
Author: Dan Smith <email address hidden>
Date: Fri Apr 4 08:38:01 2014 -0700

    Revert object-assuming changes to _post_live_migration()

    Change I140bfec2a52bf659a725a7dbe78ba5c527ed26de converted the
    _post_live_migration() call to assume it was passed an object
    because the post_live_migration_at_destination() method uses them.
    However, it is also called from live_migration() on the source,
    which clearly is still using instance dicts.

    This patch reverts the changes that assume an instance object,
    as well as the tests that were changed as a result. It also
    changes the clear_events_for_instance() call to use dict syntax,
    since that is called from the live migration path currently.

    Closes-bug: #1302334
    Change-Id: I6e915035bc2c1b890e606d17a4195b88c3f26b13

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (milestone-proposed)

Reviewed: https://review.openstack.org/85421
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b5280671768bdf861d9f72f014a83bb07af09ab6
Submitter: Jenkins
Branch: milestone-proposed

commit b5280671768bdf861d9f72f014a83bb07af09ab6
Author: Dan Smith <email address hidden>
Date: Fri Apr 4 08:38:01 2014 -0700

    Revert object-assuming changes to _post_live_migration()

    Change I140bfec2a52bf659a725a7dbe78ba5c527ed26de converted the
    _post_live_migration() call to assume it was passed an object
    because the post_live_migration_at_destination() method uses them.
    However, it is also called from live_migration() on the source,
    which clearly is still using instance dicts.

    This patch reverts the changes that assume an instance object,
    as well as the tests that were changed as a result. It also
    changes the clear_events_for_instance() call to use dict syntax,
    since that is called from the live migration path currently.

    (cherry picked from commit 887d214186631d3d1f49451fa72feb11251d2176)
    Change-Id: I6e915035bc2c1b890e606d17a4195b88c3f26b13
    Closes-bug: #1302334

Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-rc2 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.