'NoneType' object has no attribute 'address_scope_id'

[Aurelien Lourot]

In this test run [0] we deploy [1] OpenStack Ussuri on Ubuntu focal (20.04) with Neutron plugin 'ovs'. Creating an instance failed. nova-compute reported:

Failed to build and run instance: nova.exception.PortBindingFailed: Binding failed for port c2e062b7-0dbb-458e-87b5-4eef5930f1f1, please check neutron logs for more information.

neutron-server's logs show:

Error during notification for neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin.port_callback-433877 port, after_update: AttributeError: 'NoneType' object has no attribute 'address_scope_id'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
    callback(resource, event, trigger, **kwargs)
  File "/usr/lib/python3/dist-packages/neutron_dynamic_routing/services/bgp/bgp_plugin.py", line 368, in port_callback
    ext_nets = self.get_external_networks_for_port(ctx,
  File "/usr/lib/python3/dist-packages/neutron_dynamic_routing/db/bgp_db.py", line 1189, in get_external_networks_for_port
    ext_scope_set.add(ext_pool.address_scope_id)
AttributeError: 'NoneType' object has no attribute 'address_scope_id'
Failed to bind port c2e062b7-0dbb-458e-87b5-4eef5930f1f1 on host juju-19a362-zaza-73e874daef6c-15.project.serverstack for vnic_type normal using segments [{'id': 'ce5ad0ec-44a0-46e8-bf7f-2794b7fdb508', 'network_type': 'gre', 'physical_network': None, 'segmentation_id': 1, 'network_id': 'c1a9e451-5e90-4124-a531-79f24f1bc9e6'}]

0: https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_func_full/openstack/charm-manila-ganesha/748639/3/6889/index.html
1: https://opendev.org/openstack/charm-manila-ganesha/src/branch/master/src/tests/bundles/focal-ussuri.yaml

[Aurelien Lourot (aurelien-lourot)]

I'm able to reproduce it locally and I'm wondering how we end up running some bgp code in this setup. This actually seems to be caused by our Juju config `neutron-plugin: ovs`, which shouldn't be used on Ussuri+:

https://opendev.org/openstack/charm-neutron-api/src/branch/master/hooks/neutron_api_context.py#L567
https://opendev.org/openstack/charm-neutron-api/src/branch/master/hooks/neutron_api_context.py#L618

[Dr. Jens Harbott (j-harbott)]

This looks like a deployment issue to me, If you find a way to reproduce without using charms, please update accordingly.

[Aurelien Lourot (aurelien-lourot)]

fnordahl and I suspect that this error is actually a red herring and the real issue may be something else. We see RPC timeouts in the same timeframe as the failure. This PR adds some robustness to our tests: https://github.com/openstack-charmers/zaza-openstack-tests/pull/421