Overcloud stack creation failed: IP address 10.0.0.5 is already allocated to subnet

[Amol Kahat]

Description
===========
- Stack creation failed.
- ERROR: IpAddressAlreadyAllocatedClient: resources.ExternalPort: IP address 10.0.0.5 already allocated in subnet 9dba59d8-a77b-4694-9e8c-09f72864d8fa

Job
===
- periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-master

Logs
====
- https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-master/86099b4/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
- https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-master/051c59c/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
- https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-master/c8cff86/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

Actual Error
============
2021-05-16 19:48:18 | 2021-05-16 19:48:18.164 322067 DEBUG osc_lib.cli.client_config [-] auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('osc-lib', '2.4.0')], 'verbose_level': 1, 'deferred_help': False, 'default_domain': 'default', 'timing': False, 'inspector_api_version': '1', 'auth_url': 'https://192.168.24.2:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identity_api_version': '3', 'data_processing_api_version': '1.1', 'placement_api_version': '1.0', 'metrics_api_version': '1', 'alarming_api_version': '2', 'container_infra_api_version': '1', 'queues_api_version': '2', 'share_api_version': '2.63', 'tripleoclient_api_version': '2', 'loadbalancer_api_version': '2.0', 'key_manager_api_version': '1', 'orchestration_api_version': '1', 'event_api_version': '2', 'database_api_version': '1', 'dns_api_version': '2', 'baremetal_api_version': '1.69', 'workflow_api_version': '2', 'region_name': '', 'auth_type': 'password', 'networks': []} auth_config_hook /usr/lib/python3.6/site-packages/osc_lib/cli/client_config.py:160
2021-05-16 19:48:18 | 2021-05-16 19:48:18.165 322067 INFO osc_lib.clientmanager [-] Using auth plugin: password
2021-05-16 19:48:18 | 2021-05-16 19:48:18.165 322067 DEBUG osc_lib.clientmanager [-] Using parameters {'auth_url': 'https://192.168.24.2:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'admin', 'user_domain_name': 'Default', 'password': '***'} setup_auth /usr/lib/python3.6/site-packages/osc_lib/clientmanager.py:154
2021-05-16 19:48:18 | 2021-05-16 19:48:18.167 322067 DEBUG osc_lib.clientmanager [-] Get auth_ref auth_ref /usr/lib/python3.6/site-packages/osc_lib/clientmanager.py:202
2021-05-16 19:48:19 | 2021-05-16 19:48:19.010 322067 DEBUG heatclient.osc.v1.stack_failures.ListStackFailures [-] run(Namespace(long=False, stack='overcloud')) run /usr/lib/python3.6/site-packages/osc_lib/command/command.py:38
2021-05-16 19:48:20 | overcloud.PublicVirtualIP.ExternalPort:
2021-05-16 19:48:20 | resource_type: OS::Neutron::Port
2021-05-16 19:48:20 | physical_resource_id:
2021-05-16 19:48:20 | status: CREATE_FAILED
2021-05-16 19:48:20 | status_reason: |
2021-05-16 19:48:20 | IpAddressAlreadyAllocatedClient: resources.ExternalPort: IP address 10.0.0.5 already allocated in subnet 9dba59d8-a77b-4694-9e8c-09f72864d8fa
2021-05-16 19:48:20 | Neutron server returns request_ids: ['req-158589d0-387d-4e91-bd54-02f5b680ac6a']
2021-05-16 19:48:20 | 2021-05-16 19:48:20.311 322067 DEBUG osc_lib.shell [-] clean_up ListStackFailures: clean_up /usr/lib/python3.6/site-packages/osc_lib/shell.py:496
2021-05-16 19:48:20 | 2021-05-16 19:48:20.312 322067 INFO osc_lib.shell [-] END return value: 0
2021-05-16 19:48:20 | 2021-05-16 19:48:20.314 322067 WARNING tripleoclient.utils.safe_write [-] The output file /home/zuul/overcloud-deploy/overcloud/overcloud-deployment_status.yaml will be overriden
2021-05-16 19:48:20 | /usr/lib/python3.6/site-packages/barbicanclient/__init__.py:61: UserWarning: The secrets module is moved to barbicanclient/v1 directory, direct import of barbicanclient.secrets will be deprecated. Please import barbicanclient.v1.secrets instead.
2021-05-16 19:48:20 | % (name, name, name))
2021-05-16 19:48:20 | The output file /home/zuul/overcloud-deploy/overcloud/overcloud-deployment_status.yaml will be overriden
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: ValueError: Failed to deploy: Heat Stack update failed.
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last):
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 432, in _try_overcloud_deploy_with_compat_yaml
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud deployment_options=deployment_options)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 234, in _heat_deploy
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.log, self.working_dir)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 103, in deploy_without_plan
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise exceptions.DeploymentError("Heat Stack update failed.")
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud tripleoclient.exceptions.DeploymentError: Heat Stack update failed.
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud During handling of the above exception, another exception occurred:
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last):
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1056, in take_action
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud user_tht_root, created_env_files)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 395, in deploy_tripleo_heat_templates
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud deployment_options=deployment_options)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 435, in _try_overcloud_deploy_with_compat_yaml
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise ValueError(messages)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud ValueError: Failed to deploy: Heat Stack update failed.
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud 
2021-05-16 19:48:20 | Exception occured while running the command
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last):
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 432, in _try_overcloud_deploy_with_compat_yaml
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud deployment_options=deployment_options)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 234, in _heat_deploy
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.log, self.working_dir)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 103, in deploy_without_plan
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise exceptions.DeploymentError("Heat Stack update failed.")
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud tripleoclient.exceptions.DeploymentError: Heat Stack update failed.
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud During handling of the above exception, another exception occurred:
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last):
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1056, in take_action
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud user_tht_root, created_env_files)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 395, in deploy_tripleo_heat_templates
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud deployment_options=deployment_options)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 435, in _try_overcloud_deploy_with_compat_yaml
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise ValueError(messages)
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud ValueError: Failed to deploy: Heat Stack update failed.
2021-05-16 19:48:20 | 2021-05-16 19:48:20.316 322067 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud
2021-05-16 19:48:20 | 2021-05-16 19:48:20.336 322067 ERROR openstack [-] Failed to deploy: Heat Stack update failed.
2021-05-16 19:48:20 | Failed to deploy: Heat Stack update failed.
2021-05-16 19:48:20 | 2021-05-16 19:48:20.336 322067 DEBUG osc_lib.shell [-] clean_up DeployOvercloud: Failed to deploy: Heat Stack update failed. clean_up /usr/lib/python3.6/site-packages/osc_lib/shell.py:496

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/791704

[Rabi Mishra (rabi)]

I think https://review.opendev.org/c/openstack/tripleo-quickstart/+/791704 would fix it. Can we run a test job to verify?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart/+/791704
Committed: https://opendev.org/openstack/tripleo-quickstart/commit/6dd1a9f9f9a2bc5a2e693b702b43b1aa4fc21aa2
Submitter: "Zuul (22348)"
Branch: master

commit 6dd1a9f9f9a2bc5a2e693b702b43b1aa4fc21aa2
Author: ramishra <email address hidden>
Date: Mon May 17 14:54:26 2021 +0530

    Fix featureset002 for network_provision/baremetal_provision

    Use overcloud-vips-deployed.yaml
    Not include network-isolation-absolute.yaml >= wallaby

    Change-Id: I95d47264bd9dc48d3dc6f1e98598873e7691e487
    Closes-Bug: #1928651