'PortChainContext' object has no attribute 'session': AttributeError: 'PortChainContext' object has no attribute 'session' when trying to launch kolla-ansible's sfc demo

[Mathieu Goessens]

Hi,

After performing a fresh kolla-ansible queens installation on a Ubuntu 16.04 LTS, launching the kolla-ansible tacker demo leads the vnffg to not be created, with errors in neutron logs that looks to be related sfc : https://poolp.org/~gebura/openstack/neutron-server.log.txt & https://poolp.org/~gebura/openstack/neutron-input-log.txt

How to reproduce :
- Install Openstack with Kolla-ansible on a Ubuntu 16.04 LTS, for example using https://poolp.org/~geb/kolla-install-simple.txt
- Clone and launch the kolla tacker demo https://github.com/openstack/kolla-ansible/blob/master/contrib/demos/tacker/deploy-tacker-demo-sfc
- Verify the vnffg is not created (nothing in tacker vnffg-list) and that the neutron server log contains Python stack traces (docker exec -it neutron_server less /var/log/kolla/neutron/neutron-server.log)

I can provide access to a test system to perform debugging.
I can also do a bit of debugging myself but don't know a lot about Openstack Internals (I am not sure this bug is really a sfc one ...) .

No issue using a Pike verson (if using proper git revision of the tacker-demo because of a synthax change)

Regards,
geb

[Mathieu Goessens (gebura)]

For the sake of completeness, an important thing I forgot to add : Not only the code path (see stacktrace) seems to involve ovs, but it also leads to neutron-openvswitch-agent to be in an inconsistent state.

This inconsistent however, is not fully deterministic : sometime it leads the neutron-openvswitch-agent to use 100% of a core, sometime the process and so the docker container is continuously restarted, but in both case, there is nothing really specific in the logs (This one was produced when neutron-openvswitch-agent use a full core : https://poolp.org/~gebura/openstack/neutron-openvswitch-agent.log.txt)

[Thomas Morin (tmmorin-orange)]

Here is the stack trace involved:

2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/driver_manager.py", line 101, in _call_drivers
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager getattr(driver.obj, method_name)(context)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_log/helpers.py", line 67, in wrapper
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager return method(*args, **kwargs)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/drivers/ovs/driver.py", line 887, in delete_port_chain
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager self._delete_portchain_path(context, port_chain)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_log/helpers.py", line 67, in wrapper
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager return method(*args, **kwargs)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/drivers/ovs/driver.py", line 594, in _delete_portchain_path
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager port_chain['flow_classifiers']
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/drivers/ovs/driver.py", line 577, in _delete_path_node_flowrule
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager context, node, port, pc_corr, fc_ids)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/drivers/ovs/driver.py", line 568, in _delete_path_node_port_flowrule
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager self._delete_agent_fdb_entries(context, flow_rule)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/drivers/ovs/driver.py", line 192, in _delete_agent_fdb_entries
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager self._call_on_l2pop_driver(context, "remove_fdb_entries", flow_rule)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/networking_sfc/services/sfc/drivers/ovs/driver.py", line 170, in _call_on_l2pop_driver
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager pop_ports = self._get_remote_pop_ports(context, flow_rule)
2018-05-15 16:03:43.690 24 ERROR networking_sfc.services.sfc.driver_manager File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_log/helpers.py", line ...

[Thomas Morin (tmmorin-orange)]

A backport of the (likely) fix is at https://review.openstack.org/570866

[Mathieu Goessens (gebura)]

Thanks for investigating !

I just notice that a part of the "How to reproduce" instructions leads to a 4O4 error... Correct link if that helps: https://poolp.org/~gebura/openstack/kolla-install-simple.txt

For the sake of the completeness the bug seems not present using a devstack based installation (even if targeting master/queens).

Thanks !

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/570866
Committed: https://git.openstack.org/cgit/openstack/networking-sfc/commit/?id=ce9eef3ee315bf54d403e174e28eb2445b852d2b
Submitter: Zuul
Branch: stable/queens

commit ce9eef3ee315bf54d403e174e28eb2445b852d2b
Author: chenyaguang <email address hidden>
Date: Fri Mar 30 17:22:10 2018 +0800

    Delete port chain failed

    Replace the use of self.context (which has no session attribute),
    by self.admin_context which has one.

    Related-Bug: 1771372
    Change-Id: I71a45bc4ee450c5eaf50feb0a5594dd05ca997b5
    (cherry picked from commit ce07e3d7b97acfcf502928c3be3bee4f452abbd1)