migration rollback: driver_detach called with wrong number of args

[Steve Noyes]

I hit an extra exception during a failed live migration rollback. This exception occurred on the destination node. This bug is not about the reason for the failed migration, but for the failure in the rollback.

The exception:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 157, in _process_incoming
    res = self.dispatcher.dispatch(message)
  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)
  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
    result = func(ctxt, **new_args)
  File "/opt/stack/nova/nova/exception_wrapper.py", line 77, in wrapped
    function_name, call_dict, binary)
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/opt/stack/nova/nova/exception_wrapper.py", line 68, in wrapped
    return f(self, context, *args, **kw)
  File "/opt/stack/nova/nova/compute/manager.py", line 5090, in remove_volume_connection
    driver_bdm.driver_detach(context, instance, connector,
TypeError: driver_detach() takes exactly 5 arguments (6 given)

In compute manager.remove_volume_connection, this line calls driver_detach:

            driver_bdm.driver_detach(context, instance, connector,
                                     self.volume_api, self.driver)

This is driver_detach in block_device.DriverVolumeBlockDevice:

    def driver_detach(self, context, instance, volume_api, virt_driver):

The connector seems to be the culprit.

ubuntu xenial/libvirt/pike...

commit c2c6960e374351b3ce1b43a564b57e14b54c4877
Merge: 8d9eb67 95c190c
Author: Jenkins <email address hidden>
Date: Mon May 15 17:45:36 2017 +0000

    Merge "Make discover_hosts only query for unmapped ComputeNode records"

[Matt Riedemann (mriedem)]

This is the call from the compute manager:

https://github.com/openstack/nova/blob/c2c6960e374351b3ce1b43a564b57e14b54c4877/nova/compute/manager.py#L5087

This is the driver_detach method in the DriverVolumeBlockDevice:

https://github.com/openstack/nova/blob/c2c6960e374351b3ce1b43a564b57e14b54c4877/nova/virt/block_device.py#L264

Those args are indeed wrong and connector is missing. This is due to this refactor:

https://review.openstack.org/#/c/439520/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/469740

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/469740
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=f3454e95907c167bbbb44afd5cad491bae8cbcf2
Submitter: Jenkins
Branch: master

commit f3454e95907c167bbbb44afd5cad491bae8cbcf2
Author: Matt Riedemann <email address hidden>
Date: Wed May 31 23:58:39 2017 -0400

    Fix call to driver_detach in remove_volume_connection

    DriverVolumeBlockDevice.driver_detach is being called with
    the wrong number of arguments - there is no 'connector'
    argument in that method. This is a result of refactoring
    this code in I7a53e08f3fad6abb27a1d8ad425b4f916341cab3.

    The related unit test is pretty brittle so it's easy to
    see how this was missed in testing, plus we don't have
    integration testing for live migration rollbacks.

    This change fixes the call and makes the test a bit less
    brittle, but would be cleaner if it could use autospec
    although figuring out how to make that work eludes me.

    Change-Id: I12088f57f6228460b4810d39e9fca93bd589d70b
    Closes-Bug: #1694535

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0b2 development milestone.