periodic-tripleo-standalone-upgrade-stein fails upgrade repo-setup task: Check DLRN hash newest format

[Marios Andreou]

periodic-tripleo-ci-centos-7-standalone-upgrade-stein job was added to promotion critieria in [1] but there are multiple fails in [2][3][4][5] after a successful deployment fails before starting upgrade in repo setup with trace like:

         2019-05-15 16:53:27.361103 | primary | TASK [repo-setup : Check DLRN hash newest format] ******************************
        2019-05-15 16:53:27.383167 | primary | Wednesday 15 May 2019 16:53:27 +0000 (0:00:00.092) 0:00:02.821 *********
        2019-05-15 16:53:27.399729 | primary | [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
        2019-05-15 16:53:27.399889 | primary | `result|match` instead use `result is match`. This feature will be removed in
        2019-05-15 16:53:27.400010 | primary | version 2.9. Deprecation warnings can be disabled by setting
        2019-05-15 16:53:27.400088 | primary | deprecation_warnings=False in ansible.cfg.
        2019-05-15 16:53:27.405799 | primary | fatal: [undercloud]: FAILED! => {}
        2019-05-15 16:53:27.405839 | primary |
        2019-05-15 16:53:27.405884 | primary | MSG:
        2019-05-15 16:53:27.405904 | primary |
        2019-05-15 16:53:27.406348 | primary | The conditional check 'not dlrn_hash_tag_newest | match("[a-zA-Z0-9]{40}_[a-zA-Z0-9]{8}")' failed. The error was: Unexpected templating type error occurred on ({% if not dlrn_hash_tag_newest | match("[a-zA-Z0-9]{40}_[a-zA-Z0-9]{8}") %} True {% else %} False {% endif %}): expected string or buffer
        2019-05-15 16:53:27.406803 | primary |
        2019-05-15 16:53:27.407069 | primary | The error appears to have been in '/home/zuul/src/opendev.org/openstack/tripleo-quickstart/roles/repo-setup/tasks/get-dlrn-hash-newest.yml': line 2, column 3, but may
        2019-05-15 16:53:27.407184 | primary | be elsewhere in the file depending on the exact syntax problem.

This blocks stein promotion as [1] added the job into criteria.

[1] https://review.rdoproject.org/r/#/c/20720/
[2] https://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-upgrade-stein/0dc0ef1/job-output.txt.gz#_2019-05-15_16_53_27_361103
[3] https://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-upgrade-stein/92cd208/job-output.txt.gz#_2019-05-19_10_32_53_398570
[4] https://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-upgrade-stein/0241888/job-output.txt.gz#_2019-05-19_22_37_40_080363
[5] https://trunk-primary.rdoproject.org/api-centos-stein/api/civotes_detail.html?ci_name=periodic-tripleo-ci-centos-7-standalone-upgrade-stein

[Marios Andreou (marios-b)]

related revert posted at https://review.rdoproject.org/r/#/c/20833/ to remove standalone-upgrade-stein from criteria

[Marios Andreou (marios-b)]

don't know why yet but dlrn_hash_tag_newest is not being picked up in https://github.com/openstack/tripleo-quickstart/blob/ea6191e46d7ffd22c791f8eddf4da675e3c4f704/roles/repo-setup/tasks/get-dlrn-hash-newest.yml#L6 even though it looks like a related "dlrn_hash_newest" (no 'tag') is passed in with

2019-05-15 16:53:21.779332 | primary | +(./toci_quickstart.sh:145): /home/zuul/workspace/.quickstart/bin/ansible-playbook --tags build,standalone,standalone-upgrade --extra-vars @/home/zuul/workspace/.quickstart/config/release/tripleo-ci/CentOS-7/promotion-testing-hash-stein.yml -e dlrn_hash=b53ee474664be57e18f55866d294115c69a0b611_ce32306b -e get_build_command=b53ee474664be57e18f55866d294115c69a0b611_ce32306b -e dlrn_hash_newest=28a341acfd77fe34dc8948ddb663439f67dea7d6_4b08f8e2 --extra-vars @/home/zuul/src/opendev.org/openstack/tripleo-quickstart/config/general_config/featureset-multinode-common.yml --extra-vars @/home/zuul/src/opendev.org/openstack/tripleo-quickstart/config/general_config/featureset056.yml --extra-vars @/home/zuul/src/opendev.org/openstack/tripleo-ci/toci-quickstart/config/testenv/standalone.yml --extra-vars @/home/zuul/src/opendev.org/openstack/tripleo-ci/toci-quickstart/config/testenv/standalone-rdocloud.yml -e toci_vxlan_networking=false -e vxlan_networking=false --extra-vars local_working_dir=/home/zuul/workspace/.quickstart --extra-vars virthost=127.0.0.2 --inventory /home/zuul/workspace/.quickstart/hosts --extra-vars tripleo_root=/home/zuul/src/opendev.org/openstack --extra-vars working_dir=/home/zuul --skip-tags tripleo-validations,teardown-all --extra-vars @/home/zuul/workspace/logs/zuul-variables.yaml /home/zuul/workspace/.quickstart/playbooks/multinode-standalone-upgrade.yml

above like dlrn_hash_newest=28a341acfd77fe34dc8948ddb663439f67dea7d6_4b08f8e2 - but I don't see where dlrn_hash_newest becomes dlrn_hash_tag_newest

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/660150

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/660150
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=c87067748e5cfbd7496268549cd9b0140b94a528
Submitter: Zuul
Branch: master

commit c87067748e5cfbd7496268549cd9b0140b94a528
Author: Wes Hayutin <email address hidden>
Date: Mon May 20 09:30:24 2019 -0600

    make sure dlrn_hash_tag_newest is defined in promotion jobs

    Match the settings betweeen check and promotion jobs
    for upgrades.

    Closes-Bug: #1829722
    Change-Id: I3e2954734152751506eac74d38b64b55727287f3

[Marios Andreou (marios-b)]

no runs yet today https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-standalone-upgrade-stein let's see

[Marios Andreou (marios-b)]

OK so it seems that https://review.opendev.org/660150 gets us passed the dlrn_hash_newest issue which is what this bug is about so going to close this.

However the job is still failing, now on the containers tag - like

 http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-upgrade-stein/561fa01/logs/undercloud/home/zuul/standalone_upgrade.log.txt.gz

2019-05-21 22:54:41 | 2019-05-21 22:54:41.403 115041 ERROR tripleoclient.v1.tripleo_upgrade.Upgrade [-] Exception: Not found image: docker://docker.io/tripleostein/centos-binary-cinder-api:tripleo-ci-testing: ImageNotFoundException: Not found image: docker://docker.io/tripleostein/centos-binary-cinder-api:tripleo-ci-testing

I think we need a new bug for that one going to file in a bit closing this out

[Marios Andreou (marios-b)]

filed https://bugs.launchpad.net/tripleo/+bug/1829976 for comment #6 moving this fix-released move back if you disagree thanks