Overcloud deploy failed: The Workflow (tripleo.storage.v1.ceph-install) could not be found.

[Yurii Prokulevych]

Job gate-tripleo-ci-centos-7-container-to-container-upgrades-master-nv failed to deploy overcloud:

2018-04-30 16:58:43 | Deploying templates in the directory /tmp/tripleoclient-g_xx8B/tripleo-heat-templates
2018-04-30 16:58:43 |
2018-04-30 16:58:43 | Stack overcloud/5b494b00-f5ea-4e13-b249-aeb98ba1d3d3 CREATE_FAILED
2018-04-30 16:58:43 |
2018-04-30 16:58:43 | overcloud.AllNodesDeploySteps:
2018-04-30 16:58:43 | resource_type: OS::TripleO::PostDeploySteps
2018-04-30 16:58:43 | physical_resource_id:
2018-04-30 16:58:43 | status: CREATE_FAILED
2018-04-30 16:58:43 | status_reason: |
2018-04-30 16:58:43 | resources.AllNodesDeploySteps: Property error: resources.WorkflowTasks_Step2.properties.tasks[0].workflow: Error validating value 'tripleo.storage.v1.ceph-install': The Workflow (tripleo.storage.v1.ceph-install) could not be found.
2018-04-30 16:58:43 | + status_code=1

Seems like it was removed in this commit:
https://github.com/openstack/tripleo-heat-templates/commit/5a4abeeeb94e86b772b0322246c1665ede21c090#diff-f84644f1b5951f535bc7c3f4151a9a90

Link to failed job: https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-container-to-container-upgrades-master-nv/2658/console

[John Fulton (jfulton-org)]

Do we restore workflow to make the CI job pass? https://review.openstack.org/#/c/565580

OR

Do we re-write upgrade job to use new ceph upgrade command? https://review.openstack.org/#/c/560423

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/565858

[John Fulton (jfulton-org)]

trown pointed out that this is happening because we use queens THT, but have master tripleo-common on the undercloud.

Thus, https://review.openstack.org/#/c/565580 should address this bug and we'll prove it works by seeing the following, which depends on it, run the same job:

 https://review.openstack.org/#/c/565858/

This type of thing may come up again unless a queens version of TripleO common can be used in this job.

[John Fulton (jfulton-org)]

I recommend we merge https://review.openstack.org/#/c/565580 for now to help this bug along.

As per https://review.openstack.org/#/c/565858, the restore helps Ceph but there are other things that failed after Ceph was deployed so this bug will require more fixes.

We can decide what to do long term with respect to restoring things that were cleaned out later and I started a thread on it: http://lists.openstack.org/pipermail/openstack-dev/2018-May/130170.html

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565580
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=b2978e36e2294b1928cdfbf2b0e64d5b30dd668f
Submitter: Zuul
Branch: master

commit b2978e36e2294b1928cdfbf2b0e64d5b30dd668f
Author: John Fulton <email address hidden>
Date: Tue May 1 20:37:10 2018 +0000

    Revert: "Remove ceph-ansible workflow."

    This reverts commit 45a296ccacc5cdd8641ca6f5794ca868a3e4162f.

    Related-Bug: #1768116
    Change-Id: I726b28dccd21b67e31e0a205e9ee5f537ce3d4db

[OpenStack Infra (hudson-openstack)]

Change abandoned by John Fulton (<email address hidden>) on branch: master
Review: https://review.openstack.org/565858
Reason: This was only made to test a change which has since merged: https://review.openstack.org/#/c/565580/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/604783

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/604783
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=1afd1f093ca15ad946480b98cd6dd79cca46c5a7
Submitter: Zuul
Branch: master

commit 1afd1f093ca15ad946480b98cd6dd79cca46c5a7
Author: John Fulton <email address hidden>
Date: Mon Sep 24 09:26:53 2018 -0400

    Remove ceph-ansible Mistral workflow

    The ceph-ansible playbook is now handled by external_deploy_tasks
    instead of workflow_tasks.

    Change-Id: I3eeb09a92e26d4e05cfb823352570d0800345f66
    Related-Bug: #1768116