rdo-cloud, ovb jobs fail due to network. Failed connect to 169.254.169.254:80; Connection refused

Bug #1790987 reported by wes hayutin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

https://logs.rdoproject.org/77/600277/1/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/08ba381/job-output.txt.gz

2018-09-05 23:37:49.383760 | tripleo-ovb-centos-7 | ++(/opt/stack/new/tripleo-ci/toci_gate_test.sh:128): python -c 'import json, sys; print json.load(sys.stdin)["uuid"]'
2018-09-05 23:37:49.387193 | tripleo-ovb-centos-7 | % Total % Received % Xferd Average Speed Time Time Time Current
2018-09-05 23:37:49.387782 | tripleo-ovb-centos-7 | Dload Upload Total Spent Left Speed
2018-09-05 23:37:49.388992 | tripleo-ovb-centos-7 |
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to 169.254.169.254:80; Connection refused
2018-09-05 23:37:49.393667 | tripleo-ovb-centos-7 | Traceback (most recent call last):
2018-09-05 23:37:49.393772 | tripleo-ovb-centos-7 | File "<string>", line 1, in <module>
2018-09-05 23:37:49.393940 | tripleo-ovb-centos-7 | File "/usr/lib64/python2.7/json/__init__.py", line 290, in load
2018-09-05 23:37:49.394539 | tripleo-ovb-centos-7 | **kw)
2018-09-05 23:37:49.394676 | tripleo-ovb-centos-7 | File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
2018-09-05 23:37:49.394831 | tripleo-ovb-centos-7 | return _default_decoder.decode(s)
2018-09-05 23:37:49.394951 | tripleo-ovb-centos-7 | File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
2018-09-05 23:37:49.395551 | tripleo-ovb-centos-7 | obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2018-09-05 23:37:49.395680 | tripleo-ovb-centos-7 | File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
2018-09-05 23:37:49.395847 | tripleo-ovb-centos-7 | raise ValueError("No JSON object could be decoded")
2018-09-05 23:37:49.395967 | tripleo-ovb-centos-7 | ValueError: No JSON object could be decoded
2018-09-05 23:37:49.398323 | tripleo-ovb-centos-7 | +(/opt/stack/new/tripleo-ci/toci_gate_test.sh:128): UCINSTANCEID=
2018-09-05 23:37:49.399196 | tripleo-ovb-centos-7 | ERROR: the main setup script run by this job failed - exit code: 1
2018-09-05 23:37:49.399309 | tripleo-ovb-centos-7 | please look at the relevant log files to determine the root cause
2018-09-05 23:37:49.399361 | tripleo-ovb-centos-7 | Running devstack worlddump.py
2018-09-05 23:37:49.473232 | tripleo-ovb-centos-7 | /bin/sh: brctl: command not found
2018-09-05 23:37:50.462895 | tripleo-ovb-centos-7 | Cleaning up host
2018-09-05 23:37:50.463058 | tripleo-ovb-centos-7 | ... this takes 3 - 4 minutes (logs at logs/devstack-gate-cleanup-host.txt.gz)
2018-09-05 23:38:08.179403 | tripleo-ovb-centos-7 | [WARNING]: Could not match supplied host pattern, ignoring: subnodes
2018-09-05 23:38:08.179635 | tripleo-ovb-centos-7 | [WARNING]: No hosts matched, nothing to do
2018-09-05 23:38:10.836776 | tripleo-ovb-centos-7 | Done.
2018-09-05 23:38:12.233636 | tripleo-ovb-centos-7 | *** FAILED with status: 1
2018-09-05 23:38:12.508264 | tripleo-ovb-centos-7 | ERROR
2018-09-05 23:38:12.508575 | tripleo-ovb-centos-7 | {
2018-09-05 23:38:12.508653 | tripleo-ovb-centos-7 | "delta": "0:03:18.263318",
2018-09-05 23:38:12.508746 | tripleo-ovb-centos-7 | "end": "2018-09-05 23:38:12.235206",
2018-09-05 23:38:12.508832 | tripleo-ovb-centos-7 | "msg": "non-zero return code",
2018-09-05 23:38:12.508884 | tripleo-ovb-centos-7 | "rc": 1,
2018-09-05 23:38:12.508946 | tripleo-ovb-centos-7 | "start": "2018-09-05 23:34:53.971888"
2018-09-05 23:38:12.509028 | tripleo-ovb-centos-7 | }

wes hayutin (weshayutin)
tags: added: alert promotion-blocker
Revision history for this message
Javier Peña (jpena-c) wrote :

I have discussed this same issue today with the RDO Cloud ops team, due to another CI job having different problems. It is fixed now, I have tested it and VMs in the rdo-cloud-tripleo tenant have access to the metadata. Can you check?

Revision history for this message
Quique Llorente (quiquell) wrote :
Revision history for this message
Alex Schultz (alex-schultz) wrote :

Closing as the previous comments indicated it is now fixed. Feel free to reopen if it occurs again.

Changed in tripleo:
milestone: rocky-rc2 → stein-1
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.