Overcloud deploy fails in scenario jobs - "no id" for containers

Bug #1802265 reported by Sagi (Sergey) Shnaidman on 2018-11-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Critical
Unassigned

Bug Description

In all scenario jobs overcloud deployment fails, messages are different, but mostly because "no id" for some container:

http://logs.openstack.org/98/604298/74/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/3a844f7/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-08_02_59_44
2018-11-08 02:59:43 | fatal: [centos-7-ovh-bhs1-0000412455]: FAILED! => {"changed": true, "cmd": "docker tag 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:current-tripleo-updated-20181108001130 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:pcmklatest", "delta": "0:00:00.031695", "end": "2018-11-08 02:59:25.283962", "msg": "non-zero return code", "rc": 1, "start"Exception occured while running the command
2018-11-08 02:59:43 | Traceback (most recent call last):
2018-11-08 02:59:43 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 28, in run
2018-11-08 02:59:43 | super(Command, self).run(parsed_args)
2018-11-08 02:59:43 | File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
2018-11-08 02:59:43 | return super(Command, self).run(parsed_args)
2018-11-08 02:59:43 | File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
2018-11-08 02:59:43 | return_code = self.take_action(parsed_args) or 0
2018-11-08 02:59:43 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 928, in take_action
2018-11-08 02:59:43 | verbosity=self.app_args.verbose_level)
2018-11-08 02:59:43 | File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/deployment.py", line 306, in config_download
2018-11-08 02:59:43 | raise exceptions.DeploymentError("Overcloud configuration failed.")
2018-11-08 02:59:43 | DeploymentError: Overcloud configuration failed.

http://logs.openstack.org/98/604298/74/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/ecb6a02/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-08_02_04_27
2018-11-08 02:04:27 | fatal: [centos-7-rax-iad-0000412459]: FAILED! => {"changed": true, "cmd": "docker tag 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:current-tripleo-updated-20181108001837 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:pcmklatest", "delta": "0:00:00.039849", "end": "2018-11-08 02:04:20.631821", "msg": "non-zero return code", "rc": 1, "start": "2018-11-08 02:04:20.591972", "stderr": "Error response from daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:current-tripleo-updated-20181108001837", "stderr_lines": ["Error response from daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:current-tripleo-updated-20181108001837"], "stdout": "", "stdout_lines": []}

http://logs.openstack.org/98/604298/74/check/tripleo-ci-centos-7-scenario003-multinode-oooq-container/89a6466/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
2018-11-08 01:45:27 | "Error: Evaluation Error: Error while evaluating a Resource Statement,

Duplicate declaration: Class[Designate::Db] is already declared; cannot redeclare at /etc/puppet/modules/tripleo/manifests/profile/base/designate/central.pp:58 at /etc/puppet/modules/tripleo/manifests/profile/base/designate/central.pp:58:5 on node centos-7-vexxhost-sjc1-0000412498.localdomain",

2018-11-08 01:45:34 | =============================================================================== Exception occured while running the command
2018-11-08 01:45:34 | Traceback (most recent call last):
2018-11-08 01:45:34 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 28, in run
2018-11-08 01:45:34 | super(Command, self).run(parsed_args)
2018-11-08 01:45:34 | File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
2018-11-08 01:45:34 | return super(Command, self).run(parsed_args)
2018-11-08 01:45:34 | File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
2018-11-08 01:45:34 | return_code = self.take_action(parsed_args) or 0
2018-11-08 01:45:34 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 928, in take_action
2018-11-08 01:45:34 | verbosity=self.app_args.verbose_level)
2018-11-08 01:45:34 | File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/deployment.py", line 306, in config_download
2018-11-08 01:45:34 | raise exceptions.DeploymentError("Overcloud configuration failed.")
2018-11-08 01:45:34 | DeploymentError: Overcloud configuration failed.
2018-11-08 01:45:34 | Overcloud configuration failed.

http://logs.openstack.org/98/604298/74/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/7f4b42b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-08_01_50_55

2018-11-08 01:50:55 | fatal: [centos-7-inap-mtl01-0000412462]: FAILED! => {"changed": true, "cmd": "docker tag 192.168.24.1:8787/tripleomaster/centos-binary-manila-share:current-tripleo-updated-20181108000959 192.168.24.1:8787/tripleomaster/centos-binary-manila-share:pcmklatest", "delta": "0:00:00.030324", "end": "2018-11-08 01:50:47.136684", "msg": "non-zero return code", "rc": 1, "start": "2018-11-08 01:50:47.106360", "stderr": "Error response from daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-manila-share:current-tripleo-updated-20181108000959", "stderr_lines": ["Error response fromException occured while running the command
2018-11-08 01:50:55 | Traceback (most recent call last):
2018-11-08 01:50:55 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 28, in run
2018-11-08 01:50:55 | super(Command, self).run(parsed_args)
2018-11-08 01:50:55 | File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
2018-11-08 01:50:55 | return super(Command, self).run(parsed_args)
2018-11-08 01:50:55 | File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
2018-11-08 01:50:55 | return_code = self.take_action(parsed_args) or 0
2018-11-08 01:50:55 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 928, in take_action
2018-11-08 01:50:55 | verbosity=self.app_args.verbose_level)
2018-11-08 01:50:55 | File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/deployment.py", line 306, in config_download
2018-11-08 01:50:55 | raise exceptions.DeploymentError("Overcloud configuration failed.")
2018-11-08 01:50:55 | DeploymentError: Overcloud configuration failed.
2018-11-08 01:50:55 | Overcloud configuration failed.
2018-11-08 01:50:55 | END return value: 1
2018-11-08 01:50:55 | daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-manila-share:current-tripleo-updated-20181108000959"], "stdout": "", "stdout_lines": []}

http://logs.openstack.org/98/604298/74/check/tripleo-ci-centos-7-scenario007-multinode-oooq-container/967272c/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-08_02_07_54

2018-11-08 02:07:54 | fatal: [centos-7-rax-ord-0000412463]: FAILED! => {"changed": true, "cmd": "docker tag 192.168.24.1:8787/tripleomaster/centos-binary-ovn-northd:current-tripleo-updated-20181108002031 192.168.24.1:8787/tripleomaster/centos-binary-ovn-northd:pcmklatest", "delta": "0:00:00.037287", "end": "2018-11-08 02:07:42.579543", "msg": "non-zero return code", "rc": 1, "start": "2018-11-08 02:07:42.542256", "stderr": "Error response from daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-ovn-northd:current-tripleo-updated-20181108002031", "stderr_lines": ["Error response from daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-ovn-northd:current-tripleo-updated-20181108002031"], "stdout": "", "stdout_lines": []}
2018-11-08 02:07:54 |

From first job:
The image exists:

http://logs.openstack.org/98/604298/74/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/3a844f7/logs/undercloud/var/log/extra/docker/docker_allinfo.log.txt.gz
192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup current-tripleo-updated-20181108001130 b6c1a3208d6d 26 minutes ago 1.21 GB

but container doesn't exist

Tags: ci Edit Tag help
summary: - Overcloud deploy fails in scenario jobs
+ Overcloud deploy fails in scenario jobs - "no id" for containers
Alex Schultz (alex-schultz) wrote :

https://review.openstack.org/#/c/616022/ is not the breaking fix. The revert did not fix the scenario jobs. Something else broke it.

Change abandoned by Alex Schultz (<email address hidden>) on branch: master
Review: https://review.openstack.org/616565
Reason: this didn't fix the scenario jobs

Changed in tripleo:
milestone: stein-2 → stein-3
wes hayutin (weshayutin) wrote :
Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers