| 2021-08-18 14:16:27 |
Son Do Xuan |
description |
I have deployed Openstack Victoria from kolla-ansible tool and Openvswich OVN for networking. neutron-ovn-db-sync generate OVN DB error when any subnet has allocated all available IP address.
--------------------
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_pluggable_backend.py", line 140, in _ipam_allocate_ips
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise n_exc.IpAddressGenerationFailure(
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util neutron_lib.exceptions.IpAddressGenerationFailure: No more IP addresses available on network 15b718eb-4589-4b9f-8b95-aa5dc71da538.
-------------------- |
I have deployed Openstack Victoria from kolla-ansible tool and Openvswich OVN for networking. neutron-ovn-db-sync tool generate OVN DB error when any subnet (disable dhcp) has allocated all available IP address. Because neutron-ovn-db-sync tool trying recreate port "network:distributed".
--------------------
2021-07-23 14:19:47.295 1417 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [req-8fb77747-70c7-459c-bab8-97dbe07a8eba - - - - -] Starting OVNClientQosExtension
2021-07-23 14:19:47.296 1417 INFO neutron.cmd.ovn.neutron_ovn_db_sync_util [req-8fb77747-70c7-459c-bab8-97dbe07a8eba - - - - -] Sync for Northbound db started with mode : repair
2021-07-23 14:19:55.231 1417 CRITICAL neutron_ovn_db_sync_util [req-8fb77747-70c7-459c-bab8-97dbe07a8eba - - - - -] Unhandled error: neutron_lib.exceptions.IpAddressGenerationFailure: No more IP addresses available on network 15b718eb-4589-4b9f-8b95-aa5dc71da538.
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_pluggable_backend.py", line 138, in _ipam_allocate_ips
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util ip_address, subnet_id = ipam_allocator.allocate(ip_request)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/ipam/subnet_alloc.py", line 240, in allocate
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise ipam_exc.IpAddressGenerationFailureAllSubnets()
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util neutron.ipam.exceptions.IpAddressGenerationFailureAllSubnets: No more IP addresses available.
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util During handling of the above exception, another exception occurred:
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/bin/neutron-ovn-db-sync-util", line 8, in <module>
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util sys.exit(main())
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/cmd/ovn/neutron_ovn_db_sync_util.py", line 229, in main
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util synchronizer.do_sync()
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 101, in do_sync
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self.sync_networks_ports_and_dhcp_opts(ctx)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 987, in sync_networks_ports_and_dhcp_opts
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self._sync_metadata_ports(ctx, db_ports)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 912, in _sync_metadata_ports
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self._ovn_client.update_metadata_port(ctx, net['id'])
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2155, in update_metadata_port
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util update_metadata_port_fixed_ips(metadata_port,
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2122, in update_metadata_port_fixed_ips
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self._plugin.update_port(n_context.get_admin_context(),
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/common/utils.py", line 687, in inner
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return f(self, context, *args, **kwargs)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/api.py", line 233, in wrapped
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return method(*args, **kwargs)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/api.py", line 139, in wrapped
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util setattr(e, '_RETRY_EXCEEDED', True)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self.force_reraise()
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util six.reraise(self.type_, self.value, self.tb)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise value
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/api.py", line 135, in wrapped
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return f(*args, **kwargs)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_db/api.py", line 154, in wrapper
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util ectxt.value = e.inner_exc
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self.force_reraise()
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util six.reraise(self.type_, self.value, self.tb)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise value
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_db/api.py", line 142, in wrapper
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return f(*args, **kwargs)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/api.py", line 183, in wrapped
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util LOG.debug("Retry wrapper got retriable exception: %s", e)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self.force_reraise()
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util six.reraise(self.type_, self.value, self.tb)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise value
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/api.py", line 179, in wrapped
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return f(*dup_args, **dup_kwargs)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/plugins/ml2/plugin.py", line 1653, in update_port
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util updated_port = super(Ml2Plugin, self).update_port(context, id,
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/api.py", line 233, in wrapped
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return method(*args, **kwargs)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/db_base_plugin_v2.py", line 1482, in update_port
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self.ipam.update_port(
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_backend_mixin.py", line 711, in update_port
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util changes = self.update_port_with_ips(context,
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_pluggable_backend.py", line 427, in update_port_with_ips
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util changes = self._update_ips_for_port(context,
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_pluggable_backend.py", line 381, in _update_ips_for_port
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util added = self._ipam_allocate_ips(context, ipam_driver,
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_pluggable_backend.py", line 148, in _ipam_allocate_ips
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util return
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util self.force_reraise()
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util six.reraise(self.type_, self.value, self.tb)
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise value
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron/db/ipam_pluggable_backend.py", line 140, in _ipam_allocate_ips
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util raise n_exc.IpAddressGenerationFailure(
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util neutron_lib.exceptions.IpAddressGenerationFailure: No more IP addresses available on network 15b718eb-4589-4b9f-8b95-aa5dc71da538.
2021-07-23 14:19:55.231 1417 ERROR neutron_ovn_db_sync_util
-------------------- |
|
