tripleo-quickstart-promote-master-current-tripleo-delorean-minimal job runs on RDO Phase 1 in promotion pipeline, It is consistently failing from yesterday.
Below is the more details:
https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-current-tripleo-delorean-minimal-370/undercloud/home/stack/overcloud_deploy.log.gz
2018-07-12 18:09:49 | Deploying templates in the directory /tmp/tripleoclient-iplAUg/tripleo-heat-templates
2018-07-12 18:09:49 | Initializing overcloud plan deployment
2018-07-12 18:09:49 | Creating overcloud Heat stack
2018-07-12 18:09:49 | Failed to run action [action_ex_id=424fb011-75f7-4391-90f4-5b498128fe69, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'skip_deploy_identifier': False, u'container': u'overcloud', u'timeout': 90}']
2018-07-12 18:09:49 | ERROR: SSLError: : resources.Controller<nested_stack>.resources.0<https://192.168.24.2:13808/v1/AUTH_e3b4e244dcdb419782f6778f29efa3a0/overcloud/puppet/controller-role.yaml>.resources.Controller: : SSL exception connecting to https://192.168.24.3:8774/v2.1/: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')],)",)
2018-07-12 18:09:49 | + status_code=1
2018-07-12 18:09:49 | + openstack stack list
2018-07-12 18:09:49 | + grep -q overcloud
2018-07-12 18:09:57 | + echo 'overcloud deployment not started. Check the deploy configurations'
2018-07-12 18:09:57 | overcloud deployment not started. Check the deploy configurations
2018-07-12 18:09:57 | + exit 1
https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-current-tripleo-delorean-minimal-370/undercloud/var/log/extra/errors.txt.gz
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor [req-aafe3f78-270b-4f7d-b53c-91d47f0e626a 769952915fdf4012b15a3bb1c6055308 e3b4e244dcdb419782f6778f29efa3a0 - default default] Failed to run action [action_ex_id=424fb011-75f7-4391-90f4-5b498128fe69, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'skip_deploy_identifier': False, u'container': u'overcloud', u'timeout': 90}']
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor Traceback (most recent call last):
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/mistral/executors/default_executor.py", line 114, in run_action
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor result = action.run(action_ctx)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/actions/deployment.py", line 201, in run
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor return heat.stacks.create(**stack_args)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/v1/stacks.py", line 171, in create
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor data=kwargs, headers=headers)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 289, in post
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor return self.client_request("POST", url, **kwargs)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 279, in client_request
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor resp, body = self.json_request(method, url, **kwargs)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 268, in json_request
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor resp = self._http_request(url, method, **kwargs)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 231, in _http_request
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor raise exc.from_response(resp)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor HTTPBadRequest: ERROR: SSLError: : resources.Controller<nested_stack>.resources.0<https://192.168.24.2:13808/v1/AUTH_e3b4e244dcdb419782f6778f29efa3a0/overcloud/puppet/controller-role.yaml>.resources.Controller: : SSL exception connecting to https://192.168.24.3:8774/v2.1/: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')],)",)
2018-07-12 18:09:47.520 ERROR /var/log/mistral/executor.log: 21072 ERROR mistral.executors.default_executor
2018-07-12 18:04:37.097 ERROR /var/log/keystone/keystone.log: 23625 ERROR keystone.assignment.core [req-26bdd5d5-ee50-475d-ae7b-12fb69a36792 - - - - -] Circular reference found role inference rules - 871dced7d7e14137a24bc3b327e70803.
2018-07-12 18:04:41.376 ERROR /var/log/keystone/keystone.log: 23626 ERROR keystone.assignment.core [req-4b1b7e0b-b9c0-47e2-943e-8e358420
Logs from other run:
https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-current-tripleo-delorean-minimal-371/undercloud/home/stack/overcloud_deploy.log.gz
https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-current-tripleo-delorean-minimal-372/undercloud/home/stack/overcloud_deploy.log.gz
I think it was an issue with the TLS port for keystone that wasn't properly passed - but still, not 100% sure, It needs investigation
From IRC discussion related to this bug: /ci.centos. org/artifacts/ rdo/jenkins- tripleo- quickstart- promote- master- current- tripleo- delorean- minimal- 370/undercloud/ var/log/ extra/net infra/tripleo- ci master: [WIP] Use toci scripts as templates for zuulv3 https:/ /review. openstack. org/581331 tls/certs/ undercloud- 192.168. 24.2.pem according to https:/ /ci.centos. org/artifacts/ rdo/jenkins- t quickstart- promote- master- current- tripleo- delorean- minimal- 370/undercloud/ etc/haproxy/ haproxy. cfg.gz nat/redhat/ x-hvdjdyyudrkqi kff) has quit (Quit: Leaving) 37.8.44. 46) has joined #tripleo /github. com/openstack/ tripleo- quickstart/ blob/master/ config/ general_ config/ minimal. yml is used for this job
Tengu │ chkumar|ruck: guess there's no way to debug this with some running instance?
Tengu │ chkumar|ruck: might be interesting to check what's actually listening on the said ports (13808 and 8774), and if they are TLS, what's the certificate
│ content (if any)
Tengu │ chkumar|ruck: in fact.... it might even be due the lack of certificates. if a client wants to talk TLS to a non-TLS port, I think this kind of error
│ might appear.
r|ruck │ Tengu: https:/
│ stat.txt.gz
Tengu │ chkumar|ruck: ok, haproxy, of course. Will see its configuration (it's in containers right?)
gerrit │ Quique Llorente proposed openstack-
-- │ amoralej|off is now known as amoralej
◀▬▬ │ jd_ (~<email address hidden>) has quit (Ping timeout: 276 seconds)
Tengu │ bind 192.168.24.2:13808 transparent ssl crt /etc/pki/
│ ripleo-
◀▬▬ │ dmacpher (dmacpher@
Tengu │ chkumar|ruck: certificate is apparently valid, at least on the host. Will check if I find anything in container things.
▬▬▶ │ waleedm (~waleedm@
Tengu │ oh. hm. no container.
r|ruck │ yes no container,
r|ruck │ Tengu: https:/
▬▬▶ │ suuuper (~<email address hidden>) has joined #tripleo ...