Trunk port can not be deleted after removing trunk network

[daniel.pawlik]

Hi,

After removing trunk network, ports related to that network can not be deleted because Neutron API is returning an error:

Failed to delete port with name or ID '60a3c41e-b99c-4540-9302-e2322a6a474a': ConflictException: 409: Client Error for url: https://mynetwork.com/v2.0/ports/60a3c41e-b99c-4540-9302-e2322a6a474a, Port 60a3c41e-b99c-4540-9302-e2322a6a474a is currently a parent port for trunk d672597a-5e8d-4b4c-a33c-b73fcec506cb.

or

Failed to delete port with name or ID 'a95bf690-8c9b-4239-bd40-b9f01e7a0116': ConflictException: 409: Client Error for url: https://mynetwork.com/v2.0/ports/a95bf690-8c9b-4239-bd40-b9f01e7a0116, Port a95bf690-8c9b-4239-bd40-b9f01e7a0116 is currently a subport for trunk d672597a-5e8d-4b4c-a33c-b73fcec506cb.

Steps to reproduce after removing VM's with trunk interfaces:
- remove trunk network
- on removing the port associated with trunk network, Neutron is raising an error.

[Lajos Katona (lajos-katona)]

Hi, I have issues with my local env where I can try to reproduce this issue. Could you please give some details about your environment. Which branch are you using? I will try first on master. Which backend are you using? OVS/OVN?

[Lajos Katona (lajos-katona)]

Ok, I checked finally with OVS and OVN (same result), and I got this:
$ openstack network delete parent_port_net
Failed to delete network with name or ID 'parent_port_net': ConflictException: 409: Client Error for url: http://100.109.0.48:9696/networking/v2.0/networks/e4ccd55b-a4ea-47c0-b1d1-473dc4ba74a5, Unable to complete operation on network e4ccd55b-a4ea-47c0-b1d1-473dc4ba74a5. There are one or more ports still in use on the network, id for these ports is: df9734ca-9774-4816-a3bf-a4beab3d6d2f.
1 of 1 networks failed to delete.

Here's what I did:
openstack network create net1
openstack subnet create subnet1 --subnet-range 192.171.0.0/27 --network net1
openstack network create parent_port_net
openstack subnet create parent_port_subnet --subnet-range 192.172.0.0/27 --network parent_port_net
openstack port create parent_port --network parent_port_net
openstack port create subport --network net1
openstack network trunk create --parent-port parent_port --subport port=subport,segmentation-type=vlan,segmentation-id=99 my_trunk

(I got the same conflict when I tried to delete net1 which is the network of my subport)

If you have any details about your environment and how you reproduced the issue please feel free to reopen this bug report and add the necessary comments here.

[daniel.pawlik (daniel-pawlik)]

Hi,
the current backend is OVS (AFAIK).
We are using this playbook [1] for deploying the VMs in the CI + add and configure network for it.
I will add the steps later today or tomorrow.

[1] https://review.rdoproject.org/r/plugins/gitiles/config/+/refs/heads/master/playbooks/crc/bootstrap-ci-network.yml#39

[daniel.pawlik (daniel-pawlik)]

Hi, so what is what is done in the playbook:

openstack network create --mtu 1450 crc_ci_bootstrap_network_name
openstack subnet create --network crc_ci_bootstrap_network_name --subnet-range 192.168.122.0/24 --no-dhcp crc_ci_bootstrap_subnet_name
openstack router create --external-gateway public crc_ci_bootstrap_router_name
openstack router add subnet crc_ci_bootstrap_router_name crc_ci_bootstrap_subnet_name
openstack port create --network crc_ci_bootstrap_network_name --fixed-ip subnet=crc_ci_bootstrap_subnet_name,ip-address=192.168.122.11 crc_bootstrap_port_name
openstack network trunk create --parent-port crc_bootstrap_port_name zuul-ci-trunk-crc_bootstrap_trunk
openstack server add port instance_name crc_bootstrap_port_name

<here few ports are created for segmentation-id: {20,21,22}>
openstack port create --network crc_ci_bootstrap_network_name --no-fixed-ip crc_bootstrap_vlan_port_name
openstack network trunk set --subport port=crc_bootstrap_vlan_port_name,segmentation-type=vlan,segmentation-id=20 zuul-ci-trunk-crc_bootstrap_trunk
<end loop>

After all:
- remove trunk network
- on removing the port associated with trunk network, Neutron is raising an error.

Example build logs: https://review.rdoproject.org/zuul/buildset/56931cb973fe4eb18766b42c6c45e564 starting from bootstrap-ci-network.yml playbook.

I will ask the team which is developing the job to add more details here.

[Lajos Katona (lajos-katona)]

Hi,
You can check the driver like this grep:
$ grep mechanism_drivers /etc/neutron/plugins/ml2/ml2_conf.ini

it will be openvswitch for your case.

I still can't reproduce the issue, as I can't delete the network:

$ openstack network delete crc_ci_bootstrap_network_name
Failed to delete network with name or ID 'crc_ci_bootstrap_network_name': ConflictException: 409: Client Error for url: http://100.109.0.142:9696/networking/v2.0/networks/46c81a26-3af5-4328-b815-611373008130, Unable to complete operation on network 46c81a26-3af5-4328-b815-611373008130. There are one or more ports still in use on the network, id for these ports is: c77d44a3-7277-4ef2-bf8c-99d85dcc92a3,397d2829-d93e-4706-a244-ea29335b009c,17140b0f-c123-43ef-801d-b8b6c0e74dc1.
1 of 1 networks failed to delete.

[daniel.pawlik (daniel-pawlik)]

Hi,
I asked our cloud provider to add more details here (what they see in logs). Hope they will fill it soon.

[Guilherme Steinmuller Pimentel (guilhermesp)]

Hi there,

Is there really an issue here? I think the errors returned here are because the trunks -> ports are not being removed before removing a network, which is expected.

The workflow to remove these is basically:

- remove parent trunk ports
- remove ports
- remove network

I think in this case, it is not possible to remove a network in cascade without firstly removing manually its children.

[daniel.pawlik (daniel-pawlik)]

hi,
"I think the errors returned here are because the trunks -> ports are not being removed before removing a network, which is expected." - that's the issue: I was able to remove network without removing ports/un-assign ports.

Lajos Katona, if you can not reproduce that in the master branch in Neutron or even in older version, we can close the bug as invalid.

Dan

[Lajos Katona (lajos-katona)]

Daniel, guilhermesp: thanks for coming back, As I wrote you cannot delete the network of a port on master (it is true for any port, not just for the parent ports of a trunk). As you wrote the cascade deletion is not working that way.