Live migrate abort results in instance going into an error state

Bug #1815466 reported by Gary Kotton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Expired
Undecided
Unassigned

Bug Description

Only the libvirt driver supports live migration abort (and that may have some caveates if the action results in a exception). If the call takes places with a driver that does not support the opertaion then the follow stack trace appears:

nova live-migration-abort doesn’t work.

It failed with exception "NotImplementedError".

Steps:
1) Do live migration (make sure vmkernel adaptors has vmotion enabled)
2) During live-migration is going on from one cluster to another trigger command nova live-migration-abort.
3) you will see nova instance goes into ERROR state.

nova-compute error logs:
========================
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server [req-8cca230a-2ef0-4af4-9172-2259a38496ba fbb20822241440e7acac310fdc238280 bf2edf2a7ddc48a299f183b8da055c46 - d5b840b9e52c43d3ad9e208e22ad531f d5b840b9e52c43d3ad9e208e22ad531f] Exception during message handling: NotImplementedError
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 76, in wrapped
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server function_name, call_dict, binary)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server self.force_reraise()
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 67, in wrapped
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 977, in decorated_function
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 214, in decorated_function
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info())
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server self.force_reraise()
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 202, in decorated_function
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6304, in live_migration_abort
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server self.driver.live_migration_abort(instance)
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/virt/driver.py", line 943, in live_migration_abort
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server raise NotImplementedError()
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server NotImplementedError
2019-02-05 18:28:59.040 31616 ERROR oslo_messaging.rpc.server

This will resulty in the instance being in an ERROR state (although it clearly is not)

Changed in nova:
assignee: nobody → Gary Kotton (garyk)
status: New → In Progress
Changed in nova:
assignee: Gary Kotton (garyk) → Eric Fried (efried)
Changed in nova:
assignee: Eric Fried (efried) → Gary Kotton (garyk)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.opendev.org/635440

Revision history for this message
Matt Riedemann (mriedem) wrote :

Marking as incomplete because as I said on the proposed fix, I don't see how the failure in the compute manager code is setting the instance to ERROR status or reflecting anything back in the API because the API does an asynchronous RPC cast to the compute service to do the abort operation.

Changed in nova:
status: In Progress → Incomplete
assignee: Gary Kotton (garyk) → nobody
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]

Changed in nova:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.