Now, upgrade from Ocata to Pike fails at ControllerDeployment_Step1, when containers are pulled during the upgrade process:
http://logs.openstack.org/57/498057/7/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/d714b99/logs/subnode-2/var/log/messages.txt.gz#_Aug_31_23_36_11
Snippet:
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,318 ERROR: 45497 -- ERROR configuring haproxy",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,318 ERROR: 45497 -- ERROR configuring nova_placement",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,318 ERROR: 45497 -- ERROR configuring heat_api",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,318 ERROR: 45497 -- ERROR configuring nova_libvirt",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,318 ERROR: 45497 -- ERROR configuring rabbitmq",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring iscsid",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring heat",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring glance_api",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring keystone",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring nova",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring mysql",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring heat_api_cfn",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring memcached",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring crond",
Aug 31 23:36:11 centos-7-2-node-rax-ord-10727868-862288 os-collect-config: "2017-08-31 23:36:11,319 ERROR: 45497 -- ERROR configuring neutron"
AFAIK, "http: server gave HTTP response to HTTPS client" means the docker is missing the insecure registry option applied, like missing it in the config or having it in the config file, but puppet failed/ skipped/ unscheduled the docker service restart. The root cause for the latter may be the "bad" initial state of the /etc/sysconfig/ docker file so augeas fails to parse it (this can be made visible with 'puppet -tvd' or the like). What is 192.168.24.1 and how it's get configured for tripleo-ci? We need to make a local reproduce and add the aforementioned debug info.
And a registry connectivity test could help to dig some details as well:
curl -s 192.168. 24.1:8787/ v2/_catalog | python -mjson.tool
would be nice to have it in the debug results.