NotImplementedError for detach_interface in nova-compute during instance deletion

[George Shuklin]

When baremetal instance deleted there is a harmless but annoying trace in nova-compute output.

nova.compute.manager[26553]: INFO [instance: e265be67-9e87-44ea-95b6-641fc2dcaad8] Terminating instance [req-5f1eba69-239a-4dd4-8677-f28542b190bc 5a08515f35d749068a6327e387ca04e2 7d450ecf00d64399aeb93bc122cb6dae - - -]
nova.compute.resource_tracker[26553]: INFO Auditing locally available compute resources for node d02c7361-5e3a-4fdf-89b5-f29b3901f0fc [req-d34e2b7b-386f-4a3c-ae85-16860a4a9c28 - - - - -]
nova.compute.resource_tracker[26553]: INFO Final resource view: name=d02c7361-5e3a-4fdf-89b5-f29b3901f0fc phys_ram=0MB used_ram=8096MB phys_disk=0GB used_disk=480GB total_vcpus=0 used_vcpus=0 pci_stats=[] [req-d34e2b7b-386f-4a3c-ae85-16860a4a9c28 - - - - -]
nova.compute.resource_tracker[26553]: INFO Compute_service record updated for bare-compute1:d02c7361-5e3a-4fdf-89b5-f29b3901f0fc [req-d34e2b7b-386f-4a3c-ae85-16860a4a9c28 - - - - -]
nova.compute.manager[26553]: INFO [instance: e265be67-9e87-44ea-95b6-641fc2dcaad8] Neutron deleted interface 6b563aa7-64d3-4105-9ed5-c764fee7b536; detaching it from the instance and deleting it from the info cache [req-fdfeee26-a860-40a5-b2e3-2505973ffa75 11b95cf353f74788938f580e13b652d8 93c697ef6c2649eb9966900a8d6a73d8 - - -]
oslo_messaging.rpc.server[26553]: ERROR Exception during message handling [req-fdfeee26-a860-40a5-b2e3-2505973ffa75 11b95cf353f74788938f580e13b652d8 93c697ef6c2649eb9966900a8d6a73d8 - - -]
oslo_messaging.rpc.server[26553]: TRACE Traceback (most recent call last):
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
oslo_messaging.rpc.server[26553]: TRACE res = self.dispatcher.dispatch(message)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
oslo_messaging.rpc.server[26553]: TRACE return self._do_dispatch(endpoint, method, ctxt, args)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
oslo_messaging.rpc.server[26553]: TRACE result = func(ctxt, **new_args)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 75, in wrapped
oslo_messaging.rpc.server[26553]: TRACE function_name, call_dict, binary)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
oslo_messaging.rpc.server[26553]: TRACE self.force_reraise()
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
oslo_messaging.rpc.server[26553]: TRACE six.reraise(self.type_, self.value, self.tb)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 66, in wrapped
oslo_messaging.rpc.server[26553]: TRACE return f(self, context, *args, **kw)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6691, in external_instance_event
oslo_messaging.rpc.server[26553]: TRACE event.tag)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6660, in _process_instance_vif_deleted_event
oslo_messaging.rpc.server[26553]: TRACE self.driver.detach_interface(instance, vif)
oslo_messaging.rpc.server[26553]: TRACE File "/usr/lib/python2.7/dist-packages/nova/virt/driver.py", line 524, in detach_interface
oslo_messaging.rpc.server[26553]: TRACE raise NotImplementedError()
oslo_messaging.rpc.server[26553]: TRACE NotImplementedError
oslo_messaging.rpc.server[26553]: TRACE

Affected version:
nova 14.0.3
neutron 6.0.0
ironic 6.2.1

configuration for nova-compute:
compute_driver = ironic.IronicDriver

Ironic is configured to use neutron networks with generic switch as mechanism driver for ML2 pluging.

[Stephen Finucane (stephenfinucane)]

Could I just confirm that the versions are correct? Notably, the nova one (14.0.3). A quick perusal of the code suggests this shouldn't happen [1]. The linked change exists in stable/newton, from what I can tell.

  $ git branch --contains 4ad414f3
  * master
    stable/newton

[1] https://github.com/openstack/nova/commit/4ad414f3b1216393301ef268a64e61ca1a3d5be9

[George Shuklin (george-shuklin)]

It is 14.0.3. I rechecked it now: nova-14.0.3 on host, that patch is present in the code running on server. (And I restarted all novas, nova-compute, conductor and api, after upgrade).

It happens on each port. If instance have 2 ports, there are two traces.

I just reconfirm it, it still happens.

[Matt Riedemann (mriedem)]

This is separate from the detach_interface API, it's an async event from neutron that the vif was unplugged, which the compute manager tries to detach the interface, but the Ironic virt driver doesn't implement that method since it doesn't attach/detach interfaces (but plugs/unplugs vifs on server create/destroy).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/427974

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427974
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=00c0830c3e227b3a278cf10cfd68f724e433f38c
Submitter: Jenkins
Branch: master

commit 00c0830c3e227b3a278cf10cfd68f724e433f38c
Author: Matt Riedemann <email address hidden>
Date: Wed Feb 1 19:03:03 2017 -0500

    Handle NotImplementedError in _process_instance_vif_deleted_event

    The Ironic virt driver does not support attach/detach of VIFs (yet)
    so when Neutron sends an asynchronous event back to Nova that
    VIFs were unplugged on instance delete, we do not need to trace
    all over the logs when the Ironic driver does not implement
    detach_interface().

    Change-Id: I6f6533322c1179b3396e56b37c4f3f4228d8da01
    Closes-Bug: #1660317

[Jay Faulkner (jason-oldos)]

This occurred in, and was fixed in, nova. Marking invalid for Ironic.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0b1 development milestone.