nova assume ports with vnic_type=direct-physical use the libvirt interface element when processing network-vif-deleted events

[Miguel Angel Nieto Jimenez]

High level description
It should not be able to remove a port that it is attached to a VM. It is allowed to removed a port with with vnic_type=direct-physical. Exception raised:

LibvirtConfigGuestHostdevPCI accessing a non existing attribute: mac_addr

2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server [req-55778a4d-bf76-43cf-8198-02111be843d3 a82e4764680d4805b04dbc843df5ad73 233cc3938ae44e6e9fb21eaaff091999 - default default] Exception during message handling: AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server function_name, call_dict, binary)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server self.force_reraise()
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8104, in external_instance_event
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server event.tag)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8014, in _process_instance_vif_deleted_event
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server self.driver.detach_interface(context, instance, vif)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1807, in detach_interface
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server interface = guest.get_interface_by_cfg(cfg)
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 246, in get_interface_by_cfg
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server if (interface.mac_addr == cfg.mac_addr and
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
2020-02-18 17:41:20.190 8 ERROR oslo_messaging.rpc.server

Step-by-step reproduction steps
1. Create a vm with a "physical direct" port and attach to a VM

(venv) (overcloud) [stack@undercloud-0 ~]$ openstack port list | grep "50.0"
| eeb2cc67-046c-4414-a8ea-2be93bdbc542 | tempest-port-smoke-1184295488 | f8:f2:1e:03:9b:e6 | ip_address='50.0.0.110', subnet_id='a9d92b9b-e6e4-40fe-a358-e21921fdbd2d' | ACTIVE |
(venv) (overcloud) [stack@undercloud-0 ~]$ openstack port show eeb2cc67-046c-4414-a8ea-2be93bdbc542
+-----------------------+----------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+----------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | compute-1.localdomain |
| binding_profile | pci_slot='0000:05:00.3', pci_vendor_info='8086:1572', physical_network='sriov-2' |
| binding_vif_details | port_filter='False', vlan='128' |
| binding_vif_type | hostdev_physical |
| binding_vnic_type | direct-physical |
| created_at | 2020-05-07T16:08:05Z |
| data_plane_status | None |
| description | |
| device_id | 2c274d1d-b8b4-4db6-baae-b701613cc4c1 |
| device_owner | compute:nova |
| dns_assignment | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='50.0.0.110', subnet_id='a9d92b9b-e6e4-40fe-a358-e21921fdbd2d' |
| id | eeb2cc67-046c-4414-a8ea-2be93bdbc542 |
| ip_address | None |
| mac_address | f8:f2:1e:03:9b:e6 |
| name | tempest-port-smoke-1184295488 |
| network_id | c63b5f88-5447-49ab-8783-3f87a45ec9c7 |
| option_name | None |
| option_value | None |
| port_security_enabled | True |
| project_id | c2f705909fd2481b8e2dc81d9f03b29d |
| qos_policy_id | None |
| revision_number | 9 |
| security_group_ids | c230b0e7-623f-43ef-b7ac-6f40f6104736 |
| status | ACTIVE |
| subnet_id | None |
| tags | |
| trunk_details | None |
| updated_at | 2020-05-07T16:08:36Z |
+-----------------------+----------------------------------------------------------------------------------+

2. Delete the port
(venv) (overcloud) [stack@undercloud-0 ~]$ openstack port delete eeb2cc67-046c-4414-a8ea-2be93bdbc542
(venv) (overcloud) [stack@undercloud-0 ~]$ openstack port list | grep "50.0"

Apart from the error in the log file, this port is removed from the list of ports even being attached to a vm
containers/nova/nova-compute.log:2020-05-07 16:10:38.019 9 ERROR oslo_messaging.rpc.server [req-42741c25-f49b-4c3e-97ff-e5d1d6ec91d7 9c58c668079a413fb9d3e86a3b0a3a50 29f45c177bd8480a83e1037873e30a88 - default default] Exception during message handling: AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
containers/nova/nova-compute.log:2020-05-07 16:10:38.019 9 ERROR oslo_messaging.rpc.server AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'

Expected output
Port should not be removed as it is attached to a VM

Actual output
Port is removed from the port list but it continues attached to the VM.

Version
Queens

[Slawek Kaplonski (slaweq)]

That is how Neutron is designed currently. Changing that would require a lot of changes e.g. on Nova side.
IIRC we discussed that during Shanghai PTG (or in Denver) and we decided to not change that.
So from Neutron point of view You can remove port even if it's attached to the instance.

[Miguel Angel Nieto Jimenez (mnietoji)]

Then, If the behaviour is not going to be changed, I think, it should be fixed the exception:

containers/nova/nova-compute.log:2020-05-07 16:10:38.019 9 ERROR oslo_messaging.rpc.server [req-42741c25-f49b-4c3e-97ff-e5d1d6ec91d7 9c58c668079a413fb9d3e86a3b0a3a50 29f45c177bd8480a83e1037873e30a88 - default default] Exception during message handling: AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'
containers/nova/nova-compute.log:2020-05-07 16:10:38.019 9 ERROR oslo_messaging.rpc.server AttributeError: 'LibvirtConfigGuestHostdevPCI' object has no attribute 'mac_addr'

[Balazs Gibizer (balazs-gibizer)]

I suspect that the recently implemented https://blueprints.launchpad.net/nova/+spec/sriov-interface-attach-detach feature solved this issue as well as I cannot reproduce this on recent master. I will double check it with an older nova codebase..

[Balazs Gibizer (balazs-gibizer)]

I confirm this issue exists in ussuri and older branches but not on master any more.