Hi,
during upgrade of the Undercloud (Rocky containerized version) to Stein i have the following error "returned non-zero exit status 1".
When I running command:
openstack undercloud upgrade -y --no-validations --verbose --debug
I obtain:
TASK [Ensure all online data migrations for Ironic have been applied] ***********************************************************************************************************************************************************************
fatal: [iad099-undercloud]: FAILED! => {"changed": true, "cmd": "if podman ps | grep ironic_api; then\n podman exec ironic_api ironic-dbsync --config-file /etc/ironic/ironic.conf online_data_migrations\n# handle situation when container_cli is podman but\n# the containers are still under docker\nelif docker ps | grep ironic_api; then\n docker exec ironic_api ironic-dbsync --config-file /etc/ironic/ironic.conf online_data_migrations\nfi\n", "delta": "0:01:42.232913", "end": "2019-08-07 11:13:25.928615", "msg": "non-zero return code", "rc": 1, "start": "2019-08-07 11:11:43.695702", "stderr": "", "stderr_lines": [], "stdout": "23083d087659 docker.io/tripleorocky/centos-binary-ironic-api:current-tripleo \"dumb-init --singl...\" 3 months ago Up 17 hours (healthy) ironic_api", "stdout_lines": ["23083d087659 docker.io/tripleorocky/centos-binary-ironic-api:current-tripleo \"dumb-init --singl...\" 3 months ago Up 17 hours (healthy) ironic_api"]}
NO MORE HOSTS LEFT **************************************************************************************************************************************************************************************************************************
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
iad099-undercloud : ok=11 changed=1 unreachable=0 failed=1 skipped=3 rescued=0 ignored=0
Exception: Upgrade failed
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1253, in _standalone_deploy
raise exceptions.DeploymentError('Upgrade failed')
DeploymentError: Upgrade failed
None
Preserving deployment artifacts
Not cleaning working directory /home/stack/tripleo-heat-installer-templates
Not cleaning ansible directory /home/stack/undercloud-ansible-TVfj2N
Install artifact is located at /home/stack/undercloud-install-20190807091326.tar.bzip2
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Deployment Failed!
ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-8o8wzH
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Deployment failed.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 401, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
return_code = self.take_action(parsed_args) or 0
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1374, in take_action
if self._standalone_deploy(parsed_args) != 0:
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1308, in _standalone_deploy
raise exceptions.DeploymentError('Deployment failed.')
DeploymentError: Deployment failed.
clean_up Deploy: Deployment failed.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 136, in run
ret_val = super(OpenStackShell, self).run(argv)
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 281, in run
result = self.run_subcommand(remainder)
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 176, in run_subcommand
ret_value = super(OpenStackShell, self).run_subcommand(argv)
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 401, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
return_code = self.take_action(parsed_args) or 0
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1374, in take_action
if self._standalone_deploy(parsed_args) != 0:
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1308, in _standalone_deploy
raise exceptions.DeploymentError('Deployment failed.')
DeploymentError: Deployment failed.
END return value: 1
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An error has occured while deploying the Undercloud.
See the previous output for details about what went wrong.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=iad.int', '--local-ip=192.168.6.1/16', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '-y', '--upgrade', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/lifecycle/undercloud-upgrade-prepare.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/masquerade-networks.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/public-tls-undercloud.yaml', '--public-virtual-ip', '192.168.6.2', '--control-virtual-ip', '192.168.6.3', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=iad.int', '--local-ip=192.168.6.1/16', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '-y', '--upgrade', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/lifecycle/undercloud-upgrade-prepare.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/masquerade-networks.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/public-tls-undercloud.yaml', '--public-virtual-ip', '192.168.6.2', '--control-virtual-ip', '192.168.6.3', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 401, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 32, in run
super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
return_code = self.take_action(parsed_args) or 0
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/undercloud.py", line 178, in take_action
raise exceptions.DeploymentError(e)
DeploymentError: Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=iad.int', '--local-ip=192.168.6.1/16', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '-y', '--upgrade', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/lifecycle/undercloud-upgrade-prepare.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/masquerade-networks.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/public-tls-undercloud.yaml', '--public-virtual-ip', '192.168.6.2', '--control-virtual-ip', '192.168.6.3', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
clean_up UpgradeUndercloud: Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=iad.int', '--local-ip=192.168.6.1/16', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '-y', '--upgrade', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/lifecycle/undercloud-upgrade-prepare.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/masquerade-networks.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/public-tls-undercloud.yaml', '--public-virtual-ip', '192.168.6.2', '--control-virtual-ip', '192.168.6.3', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 136, in run
ret_val = super(OpenStackShell, self).run(argv)
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 281, in run
result = self.run_subcommand(remainder)
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 176, in run_subcommand
ret_value = super(OpenStackShell, self).run_subcommand(argv)
File "/usr/lib/python2.7/site-packages/cliff/app.py", line 401, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 32, in run
super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
return_code = self.take_action(parsed_args) or 0
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/undercloud.py", line 178, in take_action
raise exceptions.DeploymentError(e)
DeploymentError: Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=iad.int', '--local-ip=192.168.6.1/16', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '-y', '--upgrade', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/lifecycle/undercloud-upgrade-prepare.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/masquerade-networks.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/public-tls-undercloud.yaml', '--public-virtual-ip', '192.168.6.2', '--control-virtual-ip', '192.168.6.3', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
END return value: 1
