tripleo-ci-centos-8-content-provider job fails due to async task did not complete within the requested time

Bug #1899904 reported by Soniya Murlidhar Vyas
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Triaged
Medium
Unassigned

Bug Description

tripleo-ci-centos-8-content-provider job fails in train and ussuri because of async task did not complete within the requested time.

2020-10-15 03:44:44.081309 | primary | TASK [tripleo.operator.tripleo_container_image_build : Run tripleo container image build] ***
2020-10-15 03:44:44.081356 | primary | Thursday 15 October 2020 03:44:44 +0000 (0:00:00.566) 0:12:49.575 ******
2020-10-15 05:25:49.590488 | primary | fatal: [undercloud]: FAILED! => {
2020-10-15 05:25:49.590564 | primary | "changed": false
2020-10-15 05:25:49.590583 | primary | }
2020-10-15 05:25:49.590598 | primary |
2020-10-15 05:25:49.590613 | primary | MSG:
2020-10-15 05:25:49.590628 | primary |
2020-10-15 05:25:49.590643 | primary | async task did not complete within the requested time - 5700s
2020-10-15 05:25:49.804180 | primary |
2020-10-15 05:25:49.804298 | primary | PLAY RECAP *********************************************************************
2020-10-15 05:25:49.804320 | primary | undercloud : ok=77 changed=33 unreachable=0 failed=1 skipped=88 rescued=0 ignored=0
2020-10-15 05:25:49.804336 | primary |
2020-10-15 05:25:49.804347 | primary | Thursday 15 October 2020 05:25:49 +0000 (1:41:05.511) 1:53:55.086 ******
2020-10-15 05:25:49.804358 | primary | ===============================================================================
2020-10-15 05:25:49.804388 | primary | tripleo.operator.tripleo_container_image_build : Run tripleo container image build 6065.51s
2020-10-15 05:25:49.804399 | primary | undercloud-setup : Run the package installation script ---------------- 614.81s
2020-10-15 05:25:49.804410 | primary | repo-setup : Setup repos on live host ---------------------------------- 34.95s
2020-10-15 05:25:49.804420 | primary | build-test-packages : Ensure DLRN package dependencies ----------------- 18.08s
2020-10-15 05:25:49.804431 | primary | undercloud-setup : Install atop if configured so ----------------------- 13.66s
2020-10-15 05:25:49.804441 | primary | build-test-packages : Pip install rdopkg ------------------------------- 11.31s
2020-10-15 05:25:49.804452 | primary | container-build : Deploy the content provider registry ----------------- 11.24s
2020-10-15 05:25:49.804462 | primary | build-test-packages : Pip install DLRN ---------------------------------- 6.05s
2020-10-15 05:25:49.804472 | primary | build-test-packages : Clean up loop devices created by mock ------------- 4.87s
2020-10-15 05:25:49.804535 | primary | build-test-packages : Install and update pip ---------------------------- 4.84s
2020-10-15 05:25:49.804550 | primary | build-test-packages : Check loop devices stat --------------------------- 4.34s
2020-10-15 05:25:49.804561 | primary | container-prep : Save container_build_id on disk ------------------------ 4.04s
2020-10-15 05:25:49.804571 | primary | build-test-packages : Fetch local rdoinfo copy -------------------------- 2.69s
2020-10-15 05:25:49.804582 | primary | tripleo-inventory : regenerate ssh config for ssh connections from the virthost --- 2.27s
2020-10-15 05:25:49.804592 | primary | repo-setup : Create repo setup script ----------------------------------- 1.93s
2020-10-15 05:25:49.804603 | primary | undercloud-setup : Copy create non-root user script --------------------- 1.93s
2020-10-15 05:25:49.804613 | primary | undercloud-setup : Install package installation script ------------------ 1.90s
2020-10-15 05:25:49.804624 | primary | container-prep : Save container_build_id on disk ------------------------ 1.88s
2020-10-15 05:25:49.804634 | primary | build-test-packages : drop in the templated version of projects.ini ----- 1.88s
2020-10-15 05:25:49.804644 | primary | Create the standalone params -------------------------------------------- 1.84s
2020-10-15 05:25:49.804665 | primary | +(./toci_quickstart.sh:162): main(): exit_value=2
2020-10-15 05:25:49.804676 | primary | +(./toci_quickstart.sh:165): main(): [[ 2 == 0 ]]
2020-10-15 05:25:49.804686 | primary | +(./toci_quickstart.sh:166): main(): [[ 2 != 0 ]]
2020-10-15 05:25:49.804697 | primary | +(./toci_quickstart.sh:166): main(): echo 'Playbook run of multinode-standalone-pre.yml failed'
2020-10-15 05:25:49.804707 | primary | Playbook run of multinode-standalone-pre.yml failed
2020-10-15 05:25:49.804718 | primary | +(./toci_quickstart.sh:166): main(): break
2020-10-15 05:25:49.804728 | primary | +(./toci_quickstart.sh:168): main(): [[ 2 == 0 ]]
2020-10-15 05:25:49.804739 | primary | +(./toci_quickstart.sh:168): main(): echo 'Playbook run failed'
2020-10-15 05:25:49.804752 | primary | Playbook run failed
2020-10-15 05:25:49.804766 | primary | +(./toci_quickstart.sh:171): main(): echo 'Quickstart completed.'
2020-10-15 05:25:49.804777 | primary | Quickstart completed.
2020-10-15 05:25:49.804787 | primary | +(./toci_quickstart.sh:172): main(): exit 2
2020-10-15 05:25:50.091491 | primary | ERROR
2020-10-15 05:25:50.091839 | primary | {
2020-10-15 05:25:50.091946 | primary | "delta": "1:59:18.776593",
2020-10-15 05:25:50.092040 | primary | "end": "2020-10-15 05:25:49.806917",
2020-10-15 05:25:50.092160 | primary | "msg": "non-zero return code",
2020-10-15 05:25:50.092252 | primary | "rc": 2,
2020-10-15 05:25:50.092343 | primary | "start": "2020-10-15 03:26:31.030324"
2020-10-15 05:25:50.092433 | primary | }
2020-10-15 05:25:50.263464 |
2020-10-15 05:25:50.263688 | PLAY RECAP
2020-10-15 05:25:50.263820 | primary | ok: 8 changed: 5 unreachable: 0 failed: 1 skipped: 11 rescued: 0 ignored: 0

1. https://395c71011536eee1042b-03cab02d8b4f23536b6a5db1f6ea92b1.ssl.cf2.rackcdn.com/757605/5/check/tripleo-ci-centos-8-content-provider-train/8dcef7c/job-output.txt

2. https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b0b/757605/5/check/tripleo-ci-centos-8-content-provider-ussuri/b0b7c31/job-output.txt

Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :

Adding timestamps for logs for investigation: https://review.opendev.org/#/c/758351/

wes hayutin (weshayutin)
Changed in tripleo:
status: New → Triaged
Changed in tripleo:
milestone: victoria-rc1 → wallaby-1
Changed in tripleo:
milestone: wallaby-1 → wallaby-2
Changed in tripleo:
milestone: wallaby-2 → wallaby-3
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.