[Neutron/Nova] Need to fix attaching a vip port on VMs

[zhaobo]

From https://bugs.launchpad.net/neutron/+bug/2018529

Community had raised a fix in A release. But nova can still attach the vip port on VM without failure, even the vip port can not be used..

We can repro it in Neutron A release and OVN based deployment, relied on #2018529. I think we miss this case as following steps.
repo steps
=================
1. create a portA(we treat it as a vip port)
neutron port-show vip
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | unbound |
| binding:vnic_type | normal |
| created_at | 2024-01-16T00:24:00Z |
| description | |
| device_id | |
| device_owner | |
| dns_assignment | {"ip_address": "66.66.66.254", "hostname": "host-66-66-66-254", "fqdn": "host-66-66-66-254.zone.test."} |
| dns_domain | |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "975480c3-ed3c-4e37-a4bf-94d9aa7bf202", "ip_address": "66.66.66.254"} |
| id | bb00f200-b7d1-41d2-b132-97fd3b0eb987 |
| mac_address | fa:16:3e:52:38:d3 |
| name | vip |
| network_id | e7ad862c-bad7-4c69-8695-090be7fa9efa |
| port_security_enabled | True |
| project_id | a08affebce0540beb6d332a58f0004e8 |
| revision_number | 1 |
| security_groups | c9aca5a2-dd5e-4032-b5f1-b136864b926f |
| status | DOWN |
| tags | |
| tenant_id | a08affebce0540beb6d332a58f0004e8 |
| updated_at | 2024-01-16T00:24:00Z |
+-----------------------+-------------------------------------------------------------------------------------------------------------+

2. create another portB with address-pair portA's ip-mac.
neutron port-create testnet --allowed-address-pair ip_address=66.66.66.254,mac_address=fa:16:3e:52:38:d3 --name vip-sub
Created a new port:
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | {"mac_address": "fa:16:3e:52:38:d3", "ip_address": "66.66.66.254"} |
| binding:host_id | |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | unbound |
| binding:vnic_type | normal |
| created_at | 2024-01-16T00:25:56Z |
| description | |
| device_id | |
| device_owner | |
| dns_assignment | {"ip_address": "66.66.66.228", "hostname": "host-66-66-66-228", "fqdn": "host-66-66-66-228.zone.test."} |
| dns_domain | |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "975480c3-ed3c-4e37-a4bf-94d9aa7bf202", "ip_address": "66.66.66.228"} |
| id | 2f773e8a-51b6-4e14-9d26-bb985eec5f15 |
| mac_address | fa:16:3e:ac:b0:21 |
| name | vip-sub |
| network_id | e7ad862c-bad7-4c69-8695-090be7fa9efa |
| port_security_enabled | True |
| project_id | a08affebce0540beb6d332a58f0004e8 |
| revision_number | 1 |
| security_groups | c9aca5a2-dd5e-4032-b5f1-b136864b926f |
| status | DOWN |
| tags | |
| tenant_id | a08affebce0540beb6d332a58f0004e8 |
| updated_at | 2024-01-16T00:25:56Z |
+-----------------------+-------------------------------------------------------------------------------------------------------------+

3. create a instance X with portB(success as expect)

We see the portB had been used and updated as follow:
neutron port-show vip-sub

+-----------------------+---------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+---------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | {"mac_address": "fa:16:3e:52:38:d3", "ip_address": "66.66.66.254"} |
| binding:host_id | compute-1 |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2024-01-16T00:25:56Z |
| description | |
| device_id | 82639c68-cfc0-4441-b692-38e0f39a5232 |
| device_owner | compute:nova |
| dns_assignment | {"ip_address": "66.66.66.228", "hostname": "vip-test1", "fqdn": "vip-test1.zone.test."} |
| dns_domain | |
| dns_name | vip-test1 |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "975480c3-ed3c-4e37-a4bf-94d9aa7bf202", "ip_address": "66.66.66.228"} |
| id | 2f773e8a-51b6-4e14-9d26-bb985eec5f15 |
| mac_address | fa:16:3e:ac:b0:21 |
| name | vip-sub |
| network_id | e7ad862c-bad7-4c69-8695-090be7fa9efa |
| port_security_enabled | True |
| project_id | a08affebce0540beb6d332a58f0004e8 |
| revision_number | 4 |
| security_groups | c9aca5a2-dd5e-4032-b5f1-b136864b926f |
| status | ACTIVE |
| tags | |
| tenant_id | a08affebce0540beb6d332a58f0004e8 |
| updated_at | 2024-01-16T16:10:01Z |
+-----------------------+---------------------------------------------------------------------------------------------+

4. attach portA into instance X.

On Step 4, we saw the interface status is DOWN, but nova won't block the attach-interface request.
+------------+--------------------------------------+
| Property | Value |
+------------+--------------------------------------+
| ip_address | 66.66.66.254 |
| mac_addr | fa:16:3e:52:38:d3 |
| net_id | e7ad862c-bad7-4c69-8695-090be7fa9efa |
| port_id | bb00f200-b7d1-41d2-b132-97fd3b0eb987 |
| port_state | DOWN |
| tag | - |
+------------+--------------------------------------+

And neutron doesn't realized portA is an virtual type port, try to bind it on VM, which is not right.
neutron port-show vip

+-----------------------+---------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+---------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | compute-1 |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2024-01-16T00:24:00Z |
| description | |
| device_id | 82639c68-cfc0-4441-b692-38e0f39a5232 |
| device_owner | compute:nova |
| dns_assignment | {"ip_address": "66.66.66.254", "hostname": "vip-test1", "fqdn": "vip-test1.zone.test."} |
| dns_domain | |
| dns_name | vip-test1 |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "975480c3-ed3c-4e37-a4bf-94d9aa7bf202", "ip_address": "66.66.66.254"} |
| id | bb00f200-b7d1-41d2-b132-97fd3b0eb987 |
| mac_address | fa:16:3e:52:38:d3 |
| name | vip |
| network_id | e7ad862c-bad7-4c69-8695-090be7fa9efa |
| port_security_enabled | True |
| project_id | a08affebce0540beb6d332a58f0004e8 |
| revision_number | 3 |
| security_groups | c9aca5a2-dd5e-4032-b5f1-b136864b926f |
| status | DOWN |
| tags | |
| tenant_id | a08affebce0540beb6d332a58f0004e8 |
| updated_at | 2024-01-16T16:11:56Z |
+-----------------------+---------------------------------------------------------------------------------------------+

The reason why I leave this bug in Neutron is OVN and Neutron seems not sync the port type 'Virtual'. So Fix(https://bugs.launchpad.net/neutron/+bug/2018529) seems not work.
And OVN won't update the port type before the vip port usage from southdb to northdb. But actually, Neutron can realize the port would be Virtual type.

[Brian Haley (brian-haley)]

So this does look like a bug.

When I put a little debug in validate_port_binding_and_virtual_port() I can see the port being checked, but the port type (from OVN perspective) is not constants.LSP_TYPE_VIRTUAL, but "", so the operation succeeds.

Not exactly sure what needs to change yet.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

With an environment using master branch, the reproducer fails during the VIP port attachment to the VM. Error log: [1]

Please check that the environment you are using really has the patch [2].

Regards.

[1]https://paste.opendev.org/show/bDMjxm7tePTpmwnAIijz/
[2]https://review.opendev.org/c/openstack/neutron/+/882588

[Brian Haley (brian-haley)]

Hmm, I did a fresh deploy from master branch, unless something went wrong. Environment has since been torn down. I will let the submitter re-verify their environment has the patch, if possible.