upstream mirrors causing gate failures

[wes hayutin]

2019-11-13 21:34:50 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7-train/deps/latest/noarch/GitPython-1.0.1-5.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server"

2019-11-13 21:58:12.821739 | primary | TASK [repo-setup : Get DLRN hash - passed tag] *********************************
2019-11-13 21:58:12.862324 | primary | Wednesday 13 November 2019 21:58:12 +0000 (0:00:00.136) 0:00:17.221 ****
2019-11-13 21:58:13.574008 | primary | fatal: [subnode-1]: FAILED! => {
2019-11-13 21:58:13.574240 | primary | "changed": true,
2019-11-13 21:58:13.575685 | primary | "cmd": "set -euo pipefail\ndlrn_base=https://trunk.rdoproject.org/centos7-rocky\nif [ -e /etc/ci/mirror_info.sh ]; then\n source /etc/ci/mirror_info.sh\n NODEPOOL_RDO_PROXY=${NODEPOOL_RDO_PROXY:-https://trunk.rdoproject.org}\n dlrn_base=${dlrn_base/https:\\/\\/trunk.rdoproject.org/$NODEPOOL_RDO_PROXY}\nfi\ncurl -s --fail --show-error ${dlrn_base}/current-tripleo/delorean.repo | grep baseurl | grep -Eo '[a-zA-Z0-9]{40}_[a-zA-Z0-9]{8}'\n",
2019-11-13 21:58:13.575787 | primary | "delta": "0:00:00.105301",
2019-11-13 21:58:13.575897 | primary | "end": "2019-11-13 21:58:13.567514",
2019-11-13 21:58:13.575989 | primary | "rc": 1,
2019-11-13 21:58:13.576188 | primary | "start": "2019-11-13 21:58:13.462213"
2019-11-13 21:58:13.576255 | primary | }
2019-11-13 21:58:13.576320 | primary |
2019-11-13 21:58:13.576406 | primary | STDERR:
2019-11-13 21:58:13.576471 | primary |
2019-11-13 21:58:13.576692 | primary | curl: (52) Empty reply from server
2019-11-13 21:58:13.576734 | primary |

2019-11-13 21:59:48 | Downloading packages:
2019-11-13 21:59:53 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7/41/49/414987b15f7d79cf4c51f305589b5b6d2d8fe1b4_9e598812/os-refresh-config-10.4.1-0.20191025043248.84d8464.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server"
2019-11-13 21:59:53 | Trying other mirror.
2019-11-13 21:59:53 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7/41/49/414987b15f7d79cf4c51f305589b5b6d2d8fe1b4_9e598812/os-net-config-11.4.0-0.20191108164936.fb29607.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server"
2019-11-13 21:59:53 | Trying other mirror.
2019-11-13 21:59:54 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7-master/deps/latest/noarch/podman-manpages-1.5.1-3.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server"
2019-11-13 21:59:54 | Trying other mirror.
2019-11-13 21:59:56 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7-master/deps/latest/noarch/puppet-headless-5.5.10-5.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server"
2019-11-13 21:59:56 | Trying other mirror.
2019-11-13 21:59:58 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7-master/deps/latest/noarch/pyflakes-0.9.2-1.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server"
2019-11-13 21:59:58 | Trying other mirror.
2019-11-13 22:00:05 |
2019-11-13 22:00:05 |
2019-11-13 22:00:05 | Error downloading packages:
2019-11-13 22:00:05 | puppet-headless-5.5.10-5.el7.noarch: [Errno 256] No more mirrors to try.
2019-11-13 22:00:05 | os-refresh-config-10.4.1-0.20191025043248.84d8464.el7.noarch: [Errno 256] No more mirrors to try.
2019-11-13 22:00:05 | pyflakes-0.9.2-1.el7.noarch: [Errno 256] No more mirrors to try.
2019-11-13 22:00:05 | podman-manpages-1.5.1-3.el7.noarch: [Errno 256] No more mirrors to try.
2019-11-13 22:00:05 | os-net-config-11.4.0-0.20191108164936.fb29607.el7.noarch: [Errno 256] No more mirrors to try.

https://8804e973cb886f5af47a-3c4865f0f2de7ec21eca6cd7458b29d6.ssl.cf1.rackcdn.com/694115/1/gate/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/f139906/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz

https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/zuul_opendev_logs_af9/694120/1/gate/tripleo-ci-centos-7-containers-multinode-rocky/af9e9b7/job-output.txt

https://139a8562a978bda12faf-5ec4103c89b5d2feb0a0ca5aa5752932.ssl.cf5.rackcdn.com/694120/1/gate/tripleo-ci-centos-7-containers-multinode/32090df/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz

[Ronelle Landy (rlandy)]

This is looking better in the current gate runs:

2019-11-14 03:42:01.557224 | primary | TASK [repo-setup : Get DLRN hash - passed tag] *********************************
2019-11-14 03:42:01.593958 | primary | Thursday 14 November 2019 03:42:01 +0000 (0:00:00.132) 0:01:06.773 *****
2019-11-14 03:42:02.436234 | primary | changed: [undercloud]

[Ronelle Landy (rlandy)]

last gate failure was on 2019-11-13 23:42:27