Deployment fails with undefined variable when NetworkConfig fails

Bug #1862627 reported by Saravanan KR
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Saravanan KR

Bug Description

When the async task to run os-net-config fails, because of timeout, the deployment fails with error as undefined variable.

---------------------------------------------------------------------------------

TASK [Run NetworkConfig script] ************************************************
Monday 10 February 2020 11:44:49 +0000 (0:00:01.035) 0:03:55.515 *******

...

ok: [overcloud-computesriov-0] => {"changed": false, "failed_when_result": false, "msg": "async task did not complete within the requested time - 300s"}

TASK [Write rc of NetworkConfig script] ****************************************
Monday 10 February 2020 11:53:48 +0000 (0:08:58.565) 0:12:54.080 *******
fatal: [overcloud-computesriov-0]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'rc'\n\nThe error appears to be in '/var/lib/mistral/overcloud/deploy_steps_playbook.yaml': line 215, column 11, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Write rc of NetworkConfig script\n ^ here\n"}
changed: [overcloud-controller-0] => {"changed": true, "checksum": "b6589fc6ab0dc82cf12099d1c2d40ab994e8410c", "dest": "/var/lib/tripleo-config/os-net-config.returncode", "gid": 0, "group": "root", "md5sum": "cfcd208495d565ef66e7dff9f98764da", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:var_lib_t:s0", "size": 1, "src": "/home/tripleo-admin/.ansible/tmp/ansible-tmp-1581335628.1826398-41386609572134/source", "state": "file", "uid": 0}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
overcloud-computesriov-0 : ok=68 changed=36 unreachable=0 failed=1 skipped=11 rescued=0 ignored=0
overcloud-controller-0 : ok=52 changed=27 unreachable=0 failed=0 skipped=22 rescued=0 ignored=0
undercloud : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Monday 10 February 2020 11:53:49 +0000 (0:00:00.948) 0:12:55.029 *******
===============================================================================

Ansible failed, check log at /var/lib/mistral/overcloud/ansible.log.
Overcloud configuration failed.

Saravanan KR (skramaja)
Changed in tripleo:
milestone: none → ussuri-2
assignee: nobody → Saravanan KR (skramaja)
importance: Undecided → Medium
status: New → Triaged
Changed in tripleo:
status: Triaged → In Progress
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-2 → ussuri-3
tags: added: train-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/706878
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=f17e06a778e7663bcd88d6d83474d426f62ef6be
Submitter: Zuul
Branch: master

commit f17e06a778e7663bcd88d6d83474d426f62ef6be
Author: Saravanan KR <email address hidden>
Date: Mon Feb 10 18:30:34 2020 +0530

    Fail NetworkConfig task on timeout

    Running os-net-config as async, with failed_when as false
    results in undefined variable error if async task times out.
    Instead of ignoring failure of task, check for the
    presense of results of the command execution 'rc', if it
    is not defined, then rest of the tasks are not useful.
    Closes-Bug: #1862627

    Change-Id: Ibbcde856ac69bf73a47086d95a52c3b1a0d10911

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/716461

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/716461
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=3fff96c361a929764b337bb5784ec2efa169111c
Submitter: Zuul
Branch: stable/train

commit 3fff96c361a929764b337bb5784ec2efa169111c
Author: Saravanan KR <email address hidden>
Date: Mon Feb 10 18:30:34 2020 +0530

    Fail NetworkConfig task on timeout

    Running os-net-config as async, with failed_when as false
    results in undefined variable error if async task times out.
    Instead of ignoring failure of task, check for the
    presense of results of the command execution 'rc', if it
    is not defined, then rest of the tasks are not useful.
    Closes-Bug: #1862627

    Change-Id: Ibbcde856ac69bf73a47086d95a52c3b1a0d10911
    (cherry picked from commit f17e06a778e7663bcd88d6d83474d426f62ef6be)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 12.2.0

This issue was fixed in the openstack/tripleo-heat-templates 12.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.4.0

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.