neutron

neutron-ovn-db-sync-util fails with Traceback when notify_nova config is present

Bug #1882020 reported by Frode Nordahl on 2020-06-04

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	neutron	Fix Released	Medium	Frode Nordahl

Bug Description

If either ``notify_nova_on_port_status_changes`` or ``notify_nova_on_port_data_changes`` is set in the neutron configuration provided to the ``neutron-ovn-db-sync-util`` tool it will stop with the following Traceback on startup:

neutron-ovn-db-sync-util --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
2020-06-03 13:12:17.066 484772 INFO neutron.cmd.ovn.neutron_ovn_db_sync_util [-] Started Neutron OVN db sync
2020-06-03 13:12:17.407 484772 INFO neutron.manager [-] Loading core plugin: neutron.cmd.ovn.neutron_ovn_db_sync_util.Ml2Plugin
2020-06-03 13:12:17.412 484772 INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['geneve', 'vxlan', 'vlan', 'flat', 'local']
2020-06-03 13:12:17.417 484772 INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['physnet1']
2020-06-03 13:12:17.419 484772 INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
2020-06-03 13:12:17.421 484772 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('physnet1', [(1000, 2000)])])
2020-06-03 13:12:17.423 484772 INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan', 'vxlan']
2020-06-03 13:12:17.423 484772 INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan', 'vxlan'])
2020-06-03 13:12:17.424 484772 INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['geneve', 'vxlan', 'vlan', 'flat', 'local']
2020-06-03 13:12:17.424 484772 INFO neutron.plugins.ml2.managers [-] Configured extension driver names: []
2020-06-03 13:12:17.425 484772 INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: []
2020-06-03 13:12:17.425 484772 INFO neutron.plugins.ml2.managers [-] Registered extension drivers: []
2020-06-03 13:12:17.425 484772 INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn-sync']
2020-06-03 13:12:17.426 484772 INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn-sync']
2020-06-03 13:12:17.426 484772 INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn-sync']
2020-06-03 13:12:17.426 484772 INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
2020-06-03 13:12:17.428 484772 CRITICAL neutron_ovn_db_sync_util [-] Unhandled error: oslo_config.cfg.NoSuchOptError: no such option keystone_authtoken in group [DEFAULT]
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2197, in __getattr__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util return self._get(name)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2631, in _get
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util value, loc = self._do_get(name, group, namespace)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2649, in _do_get
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util info = self._get_opt_info(name, group)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2849, in _get_opt_info
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util raise NoSuchOptError(opt_name, group)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util oslo_config.cfg.NoSuchOptError: no such option keystone_authtoken in group [DEFAULT]
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util During handling of the above exception, another exception occurred:
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/bin/neutron-ovn-db-sync-util", line 10, in <module>
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util sys.exit(main())
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/cmd/ovn/neutron_ovn_db_sync_util.py", line 201, in main
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util manager.init()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/manager.py", line 304, in init
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util NeutronManager.get_instance()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/manager.py", line 255, in get_instance
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util cls._create_instance()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py", line 359, in inner
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util return f(*args, **kwargs)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/manager.py", line 241, in _create_instance
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util cls._instance = cls()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/manager.py", line 129, in __init__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util plugin = self._get_plugin_instance(CORE_PLUGINS_NAMESPACE,
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/manager.py", line 165, in _get_plugin_instance
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util plugin_inst = plugin_class()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/quota/resource_registry.py", line 123, in wrapper
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util return f(*args, **kwargs)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/plugin.py", line 250, in __init__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util super(Ml2Plugin, self).__init__()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/db/db_base_plugin_v2.py", line 169, in __init__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util self.nova_notifier = nova.Notifier.get_instance()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/notifiers/nova.py", line 55, in get_instance
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util cls._instance = cls()
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/neutron/notifiers/nova.py", line 59, in __init__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util auth = ks_loading.load_auth_from_conf_options(cfg.CONF, 'nova')
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/keystoneauth1/loading/conf.py", line 122, in load_from_conf_options
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util name = conf[group].auth_type
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2205, in __getitem__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util return self.__getattr__(key)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2201, in __getattr__
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util raise NoSuchOptError(name)
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util oslo_config.cfg.NoSuchOptError: no such option keystone_authtoken in group [DEFAULT]
2020-06-03 13:12:17.428 484772 ERROR neutron_ovn_db_sync_util

The tool is already overriding the ML2 AgentNotifierApi to avoid unwanted notifications sent during the sync process so I would suggest we just override the above mentioned configuration options as they will not effect the operation of the tool.

Tags:

Frode Nordahl (fnordahl) on 2020-06-04

Changed in neutron:
status:	New → In Progress
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-06-04: Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/733481

Lucas Alvares Gomes (lucasagomes) on 2020-06-05

Changed in neutron:
importance:	Undecided → Medium

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-06-05: Fix merged to neutron (master)

Reviewed: https://review.opendev.org/733481
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=559ae06bdf1f11ca861234a92434771a26ec27d8
Submitter: Zuul
Branch: master

commit 559ae06bdf1f11ca861234a92434771a26ec27d8
Author: Frode Nordahl <email address hidden>
Date: Thu Jun 4 10:35:50 2020 +0200

[OVN] Override notify_nova config in neutron-ovn-db-sync-util

    If either ``notify_nova_on_port_status_changes`` or
    ``notify_nova_on_port_data_changes`` is set in the neutron
    configuration provided to the ``neutron-ovn-db-sync-util`` tool it
    will stop with a Traceback.

    We are already overriding other parts of the ML2 Notification code
    as we do not want it executing while syncing, so let's override
    these configuration options if set as well.

Change-Id: I62765a14d376664ccc2ca3009bb9e970c9f25c03
Closes-Bug: #1882020

Changed in neutron:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-06-10: Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/734754

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-06-11: Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/734754
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1e53a770823b0053402c2fa44888ef5dd8da70c1
Submitter: Zuul
Branch: stable/ussuri

commit 1e53a770823b0053402c2fa44888ef5dd8da70c1
Author: Frode Nordahl <email address hidden>
Date: Thu Jun 4 10:35:50 2020 +0200

[OVN] Override notify_nova config in neutron-ovn-db-sync-util

    If either ``notify_nova_on_port_status_changes`` or
    ``notify_nova_on_port_data_changes`` is set in the neutron
    configuration provided to the ``neutron-ovn-db-sync-util`` tool it
    will stop with a Traceback.

    We are already overriding other parts of the ML2 Notification code
    as we do not want it executing while syncing, so let's override
    these configuration options if set as well.

    Change-Id: I62765a14d376664ccc2ca3009bb9e970c9f25c03
    Closes-Bug: #1882020
    (cherry picked from commit 559ae06bdf1f11ca861234a92434771a26ec27d8)

tags:

added: in-stable-ussuri

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.