ci.centos gate job fails at Taks: Wait for cloud-init to finish, if enabled

Bug #1913675 reported by yatin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
High
yatin

Bug Description

It fails as below:-
2021-01-28 12:45:56.219646 | 00cec312-dbfb-8f09-106f-000000000029 | RETRY | Wait for cloud-init to finish, if enabled | standalone | 4 retries left
2021-01-28 12:46:01.570955 | 00cec312-dbfb-8f09-106f-000000000029 | RETRY | Wait for cloud-init to finish, if enabled | standalone | 3 retries left
2021-01-28 12:46:06.915090 | 00cec312-dbfb-8f09-106f-000000000029 | RETRY | Wait for cloud-init to finish, if enabled | standalone | 2 retries left
2021-01-28 12:46:12.264037 | 00cec312-dbfb-8f09-106f-000000000029 | RETRY | Wait for cloud-init to finish, if enabled | standalone | 1 retries left
2021-01-28 12:46:17.616718 | 00cec312-dbfb-8f09-106f-000000000029 | FATAL | Wait for cloud-init to finish, if enabled | standalone | error={"ansible_facts": {"cloud_init_data_facts": {"status": {}}}, "attempts": 50, "changed": false, "cloud_init_data_facts": {"status": {}}}
2021-01-28 12:46:17.618451 | 00cec312-dbfb-8f09-106f-000000000029 | TIMING | Wait for cloud-init to finish, if enabled | standalone | 0:05:14.082095 | 268.28s

Example log:- https://artifacts.ci.centos.org/rdo/jenkins-tripleo-quickstart-gate-master-delorean-full-featureset052-483/undercloud/home/stack/standalone_deploy.log

In these env, cloud-init is installed and enabled but not started, leading to res.cloud_init_data_facts.status.v1.stage never get defined.

Info from system:-
[stack@undercloud ~]$ sudo systemctl status cloud-init
● cloud-init.service - Initial cloud-init job (metadata service crawler)
   Loaded: loaded (/usr/lib/systemd/system/cloud-init.service; enabled; vendor preset: disabled)
   Active: inactive (dead)
[stack@undercloud ~]$ sudo systemctl is-enabled cloud-init
enabled

"cloud-config.service": {
            "name": "cloud-config.service",
            "source": "systemd",
            "state": "inactive",
            "status": "enabled"
        },
        "cloud-final.service": {
            "name": "cloud-final.service",
            "source": "systemd",
            "state": "inactive",
            "status": "enabled"
        },
        "cloud-init-local.service": {
            "name": "cloud-init-local.service",
            "source": "systemd",
            "state": "inactive",
            "status": "enabled"
        },
        "cloud-init.service": {
            "name": "cloud-init.service",
            "source": "systemd",
            "state": "inactive",
            "status": "enabled"
        },

"res": {
        "ansible_facts": {
            "cloud_init_data_facts": {
                "status": {}
            }
        },
        "attempts": 2,
        "changed": false,
        "cloud_init_data_facts": {
            "status": {}
        },
        "failed": true
    }

yatin (yatinkarel)
Changed in tripleo:
status: New → Triaged
importance: Undecided → High
milestone: none → wallaby-2
tags: added: alert ci
Revision history for this message
Rabi Mishra (rabi) wrote :

so cloud-init services run after NetworkManager-wait-online.service and that has failed. So cloud-init did not run.

-- Logs begin at Thu 2021-01-28 11:20:18 UTC, end at Thu 2021-01-28 12:49:18 UTC. --
Jan 28 11:20:23 undercloud systemd[1]: Starting Network Manager Wait Online...
Jan 28 11:20:53 undercloud systemd[1]: NetworkManager-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Jan 28 11:20:53 undercloud systemd[1]: NetworkManager-wait-online.service: Failed with result 'exit-code'.
Jan 28 11:20:53 undercloud systemd[1]: Failed to start Network Manager Wait Online.

Revision history for this message
yatin (yatinkarel) wrote :
Changed in tripleo:
assignee: nobody → yatin (yatinkarel)
status: Triaged → In Progress
Changed in tripleo:
milestone: wallaby-2 → wallaby-3
Revision history for this message
yatin (yatinkarel) wrote :
Changed in tripleo:
milestone: wallaby-3 → wallaby-rc1
Changed in tripleo:
milestone: wallaby-rc1 → xena-1
Changed in tripleo:
milestone: xena-1 → xena-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-heat-templates (master)

Change abandoned by "yatin <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/772979
Reason: as per discussions in comments.

Changed in tripleo:
milestone: xena-2 → xena-3
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers