Undercloud minion failing validation ERROR: Heat Engine host count is 1 or less.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Fix Released
|
Critical
|
Alex Schultz |
Bug Description
Undercloud minion job is failing for master branch with validation ERROR :-
~~~
2020-07-09 04:17:30.993363 | primary | TASK [validate-minion : Execute the simple minion validation] ******************
2020-07-09 04:17:30.993383 | primary | Thursday 09 July 2020 04:17:30 +0000 (0:00:02.014) 0:51:42.505 *********
2020-07-09 04:17:49.691563 | primary | fatal: [undercloud]: FAILED! => {
2020-07-09 04:17:49.691623 | primary | "changed": false,
2020-07-09 04:17:49.691643 | primary | "cmd": "set -o pipefail && /home/zuul/
2020-07-09 04:17:49.691659 | primary | "delta": "0:00:18.236039",
2020-07-09 04:17:49.691710 | primary | "end": "2020-07-09 04:17:49.666266",
2020-07-09 04:17:49.691734 | primary | "rc": 1,
2020-07-09 04:17:49.691750 | primary | "start": "2020-07-09 04:17:31.430227"
2020-07-09 04:17:49.691765 | primary | }
2020-07-09 04:17:49.691780 | primary |
2020-07-09 04:17:49.691795 | primary | STDOUT:
2020-07-09 04:17:49.691820 | primary |
2020-07-09 04:17:49.691836 | primary | +------
2020-07-09 04:17:49.691853 | primary | | Hostname | Binary | Engine ID | Host | Topic | Updated At | Status |
2020-07-09 04:17:49.691869 | primary | +------
2020-07-09 04:17:49.691885 | primary | | undercloud.
2020-07-09 04:17:49.691900 | primary | | undercloud.
2020-07-09 04:17:49.691916 | primary | | undercloud.
2020-07-09 04:17:49.691931 | primary | | undercloud.
2020-07-09 04:17:49.691946 | primary | +------
2020-07-09 04:17:49.691962 | primary | ERROR: Heat Engine host count is 1 or less.
2020-07-09 04:17:49.691977 | primary | +------
2020-07-09 04:17:49.691992 | primary | | Hostname | Conductor Group | Alive |
2020-07-09 04:17:49.692007 | primary | +------
2020-07-09 04:17:49.692022 | primary | | undercloud.
2020-07-09 04:17:49.692036 | primary | +------
2020-07-09 04:17:49.692051 | primary | ERROR: Ironic Conductor host count is 1 or less.
2020-07-09 04:17:49.692065 | primary |
2020-07-09 04:17:49.692079 | primary |
2020-07-09 04:17:49.692094 | primary | STDERR:
2020-07-09 04:17:49.692108 | primary |
2020-07-09 04:17:49.692122 | primary | + VALIDATE_
2020-07-09 04:17:49.692137 | primary | + VALIDATE_
2020-07-09 04:17:49.692151 | primary | + RC=0
2020-07-09 04:17:49.692171 | primary | + set +x
2020-07-09 04:17:49.692186 | primary | + [[ true == \t\r\u\e ]]
2020-07-09 04:17:49.692201 | primary | + openstack orchestration service list
2020-07-09 04:17:49.692215 | primary | ++ openstack orchestration service list -c Host -f value
2020-07-09 04:17:49.692230 | primary | ++ sort
2020-07-09 04:17:49.692244 | primary | ++ uniq
2020-07-09 04:17:49.692259 | primary | ++ wc -l
2020-07-09 04:17:49.692273 | primary | + HOST_COUNT=1
2020-07-09 04:17:49.692288 | primary | + '[' 1 -le 1 ']'
2020-07-09 04:17:49.692302 | primary | + echo 'ERROR: Heat Engine host count is 1 or less.'
2020-07-09 04:17:49.692316 | primary | + RC=1
2020-07-09 04:17:49.692331 | primary | ++ openstack orchestration service list -c Status -f value
2020-07-09 04:17:49.692345 | primary | ++ grep -v up
2020-07-09 04:17:49.692360 | primary | ++ wc -l
2020-07-09 04:17:49.692374 | primary | + DOWN_STATUS=0
2020-07-09 04:17:49.692389 | primary | + '[' 0 -ge 1 ']'
2020-07-09 04:17:49.692403 | primary | + [[ true == \t\r\u\e ]]
2020-07-09 04:17:49.692417 | primary | + openstack baremetal conductor list
2020-07-09 04:17:49.692431 | primary | ++ openstack baremetal conductor list -c Hostname -f value
2020-07-09 04:17:49.692445 | primary | ++ sort
2020-07-09 04:17:49.692460 | primary | ++ uniq
2020-07-09 04:17:49.692474 | primary | ++ wc -l
2020-07-09 04:17:49.692488 | primary | + HOST_COUNT=1
2020-07-09 04:17:49.692502 | primary | + '[' 1 -le 1 ']'
2020-07-09 04:17:49.692516 | primary | + echo 'ERROR: Ironic Conductor host count is 1 or less.'
2020-07-09 04:17:49.692530 | primary | + RC=1
2020-07-09 04:17:49.692545 | primary | ++ openstack baremetal conductor list -c Alive -f value
2020-07-09 04:17:49.692559 | primary | ++ grep -v True
2020-07-09 04:17:49.692579 | primary | ++ wc -l
2020-07-09 04:17:49.692595 | primary | + DOWN_STATUS=0
2020-07-09 04:17:49.692609 | primary | + '[' 0 -ge 1 ']'
2020-07-09 04:17:49.692629 | primary | + exit 1
2020-07-09 04:17:49.692645 | primary |
2020-07-09 04:17:49.692660 | primary |
2020-07-09 04:17:49.692674 | primary | MSG:
2020-07-09 04:17:49.692717 | primary |
2020-07-09 04:17:49.692734 | primary | non-zero return code
2020-07-09 04:17:49.695032 | primary |
2020-07-09 04:17:49.695092 | primary | PLAY RECAP *******
2020-07-09 04:17:49.695113 | primary | 127.0.0.2 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
2020-07-09 04:17:49.695129 | primary | subnode-1 : ok=74 changed=29 unreachable=0 failed=0 skipped=52 rescued=0 ignored=0
2020-07-09 04:17:49.695153 | primary | undercloud : ok=108 changed=50 unreachable=0 failed=1 skipped=129 rescued=0 ignored=0
~~~
Log links:-
Failure started on this job ~14:00 UTC , 08th July
Changed in tripleo: | |
assignee: | nobody → Emilien Macchi (emilienm) |
Changed in tripleo: | |
milestone: | victoria-1 → victoria-3 |
Changed in tripleo: | |
status: | In Progress → Fix Released |
Minion deployment failed because if failed to pull containers:-
https:/ /16a586043fd88d dd8ac7- 0aa6565d1e0c06a 8a7c8c1f2b9ada9 fc.ssl. cf5.rackcdn. com/724734/ 22/check/ tripleo- ci-centos- 8-containers- undercloud- minion/ 3fe43e7/ logs/subnode- 1/home/ zuul/minion_ install. log
~~~ utils.utils [ ] Ansible execution failed. playbook: deploy_ steps_playbook. yaml, Run Status: failed, Return Code: 2 v1.tripleo_ deploy. Deploy [ ] Not cleaning working directory /home/zuul/ tripleo- heat-installer- templates v1.tripleo_ deploy. Deploy [ ] Not cleaning ansible directory /home/zuul/ minion- ansible- 82pwk13f v1.tripleo_ deploy. Deploy [ ] Install artifact is located at /home/zuul/ minion- install- 20200709041721. tar.bzip2 v1.tripleo_ deploy. Deploy [ ] !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!
2020-07-09 04:17:21.069 40274 ERROR tripleoclient.
2020-07-09 04:17:22.021 40274 WARNING tripleoclient.
2020-07-09 04:17:22.055 40274 WARNING tripleoclient.
2020-07-09 04:17:22.083 40274 WARNING tripleoclient.
2020-07-09 04:17:22.084 40274 ERROR tripleoclient.
!!!!!!!
Deployment Failed!
ERROR: Heat log files: /var/log/ heat-launcher/ undercloud_ deploy- 3w0p4527
!!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!
2020-07-09 04:17:22.085 40274 ERROR openstack [ ] Deployment failed. v1.undercloud_ minion. InstallUnderclo udMinion [ ] !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!
2020-07-09 04:17:22.290 40218 ERROR tripleoclient.
!!!!!!!
An error has occured while deploying the Undercloud Minion
See the previous output for details about what went wrong.
!!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!!!!!! !!
2020-07-09 04:17:22.291 40218 ERROR tripleoclient. v1.undercloud_ minion. InstallUnderclo udMinion [ ] Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone- role', 'UndercloudMinion', '--stack', 'minion', '-r', '/usr/share/ openstack- tripleo- heat-templates/ roles/Underclou dMinion. yaml', '--local- ip=192. 168.24. 4/24', '--templates= /usr/share/ openstack- tripleo- heat-templates' , '--networks- file=network_ data_undercloud .yaml', '-e', '/home/ zuul/tripleo- config- generated- env-files/ tripleo- undercloud- base.yaml' , '--heat-native', '-e', '/usr/share/ openstack- tripleo- heat-templates/ environments/ undercloud/ undercloud- minion. yaml', '-e', '/usr/share/ openstack- tripleo- heat-templates/ environments/ use-dns- for-vips. yaml', '-e', '/usr/share/ openstack- tripleo- heat-templates/ environments/ podman. yaml', '-e', '/home/ zuul/containers -prepare- parameter. yaml', '-e', '/usr/share/ openstack- tripleo- heat-templates/ environments/ services/ heat-engine. yaml', '-e', '/usr/share/ openstack- tripleo- heat-templates/ environments/ services/ ironic- conductor. yaml', '--deployment- user', 'zuul', '--output- dir=/home/ zuul', '-e', '/home/ zuul/tripleo- config- generated- env-files/ minion_ parameters. yaml', '--log- file=/home/ zuul/minion_ install. log', '-e', '/usr/share/ openstack- tripleo- heat-templates/ minion- stack-vstate- dropin. yaml']' returned non-zero exit status 1.
~~~
https:/ /16a586043fd88d dd8ac7- 0aa6565d1e0c06a 8a7c8c1f2b9ada9 fc.ssl. cf5.rackcdn. com/724734/ 22/check/ tripleo- ci-centos- 8-containers- undercloud- minion/ 3fe43e7/ logs/subnode- 1/home/ zuul/undercloud _minion_ install. log
...