Trunk plugin fails to push trunk object through OVO rpc endpoint on AFTER_DELETE

Bug #2065707 reported by Ihar Hrachyshka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Triaged
High
Ihar Hrachyshka

Bug Description

From here: https://0ae702c1ad12e9119b7a-757cbada98b2345898c3deef0a647dfa.ssl.cf1.rackcdn.com/917263/3/check/neutron-tempest-plugin-openvswitch/75602bd/controller/logs/screen-q-svc.txt

May 03 23:38:06.434640 np0037437196 neutron-server[58678]: DEBUG neutron.services.trunk.rpc.backend [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Emitting event after_delete for resource trunk {{(pid=58678) process_event /opt/stack/neutron/neutron/services/trunk/rpc/backend.py:66}}
May 03 23:38:06.434711 np0037437196 neutron-server[58678]: DEBUG neutron.services.trunk.rpc.server [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] neutron.services.trunk.rpc.server.TrunkStub method trunk_deleted called with arguments (<neutron_lib.context.Context object at 0x7fc48ae428f0>, <neutron.db.models_v2.Port[object at 7fc48afc6e90] {project_id='d739e50798904e7582bbda9db4cb229b', id='94c147ce-6373-4d43-8916-5fba2e878ca5', name='', network_id='b6223fc1-458a-4b09-a024-7cfc9d960208', mac_address='fa:16:3e:0b:dc:8d', admin_state_up=True, status='DOWN', device_id='', device_owner='', ip_allocation='immediate', standard_attr_id=728}>) {} {{(pid=58678) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
May 03 23:38:06.434829 np0037437196 neutron-server[58678]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Pushing event deleted for resources: {None: ['ID=94c147ce-6373-4d43-8916-5fba2e878ca5,revision_number=1']} {{(pid=58678) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}}
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Error during notification for neutron.services.trunk.rpc.backend.ServerSideRpcBackend.process_event-504527 trunk, after_delete: neutron.api.rpc.handlers.resources_rpc.InvalidResourceTypeClass: Invalid resource type None
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/neutron-lib/neutron_lib/callbacks/manager.py", line 189, in _notify_loop
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager callback.method(resource, event, trigger, payload=payload)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/neutron/neutron/services/trunk/rpc/backend.py", line 67, in process_event
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager method[event](context, payload)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager return method(*args, **kwargs)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/neutron/neutron/services/trunk/rpc/server.py", line 208, in trunk_deleted
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager self._resource_rpc.push(context, [trunk], events.DELETED)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 246, in push
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager self._push(context, resource_type, type_resources, event_type)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 250, in _push
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager _validate_resource_type(resource_type)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 52, in _validate_resource_type
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager raise InvalidResourceTypeClass(resource_type=resource_type)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager neutron.api.rpc.handlers.resources_rpc.InvalidResourceTypeClass: Invalid resource type None
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager
May 03 23:38:06.437291 np0037437196 neutron-server[58678]: DEBUG neutron_lib.callbacks.manager [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Publish callbacks [] for trunk (None), before_response {{(pid=58678) _notify_loop /opt/stack/neutron-lib/neutron_lib/callbacks/manager.py:184}}

I believe this was broken by https://review.opendev.org/c/openstack/neutron/+/885154

Changed in neutron:
assignee: nobody → Ihar Hrachyshka (ihar-hrachyshka)
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/919602

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
Bence Romsics (bence-romsics) wrote (last edit ):

I originally missed https://review.opendev.org/c/openstack/neutron/+/885154, but now that I noticed it, I think I should mention that it may have been created unnecessarily.

I'm having a hard time finding written documentation of this (api-ref, spec, devref, etc) but as my memory serves (and this could be likely found in the gazillion comments of the trunk specs) the original intention was to *allow* trunking of bound ports - as much as the implementation allowed. At the time of trunk implementation I believe all mechanism drivers, except ml2/ovs were able to create a trunk with a bound parent port. This was seen as a feature.

In ml2/ovs though, creating a trunk on a bound port would require re-plugging the vif from br-int to a trunk bridge. Which would mean data plane outage, therefore we prohibited it. Going back to the documentation again, I believe that's why we added here:

https://docs.openstack.org/api-ref/network/v2/#create-trunk

"""
Error codes:
    409 The operation returns this error code for one of these reasons:
        A system configuration prevents the operation from succeeding.
"""

If you agree, we may also consider reverting https://review.opendev.org/c/openstack/neutron/+/885154.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/919858

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/919859

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by "Ihar Hrachyshka <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/919602
Reason: Reverting the original patch that broke this: https://review.opendev.org/c/openstack/neutron/+/919858

Changed in neutron:
status: In Progress → Triaged
importance: Undecided → High
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.