Composable scenarios BM -> Containers Upgrade jobs never deploy on BM

[Martin André]

In tripleo-quickstart, we use the composable_scenario variable to indicate the path of an environment file that defines how the overcloud nodes are going to be deployed. This is working well in the general case, however in the case of the upgrade from BM to container, we need a different variable for the composable_scenario used at deploy time and the one used at upgrade time, otherwise we'll use the containers scenario to deploy the first overcloud, which means we're never actually testing BM to containers upgrade.

Here is an example where quickstart includes the '/home/jenkins/tripleo-heat-templates/ci/environments/scenario002-multinode-containers.yaml' environment file pointing to containerized resource to deploy the overcloud:

http://logs.openstack.org/57/499457/2/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container-upgrades-nv/038060a/logs/undercloud/home/jenkins/overcloud-deploy.sh

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/500546

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/500552

[Emilien Macchi (emilienm)]

I did a little bit of research and I found out that on stable/pike, the upgrade job was correctly deploying Ocata on BM and then upgrade to Pike on containers but the version of TripleO packages on Pike was too old (11 days).

I figured that tripleo-quickstart wasn't deploying current-pike repo, so I'm trying to add it here: https://review.openstack.org/500671 and test it here https://review.openstack.org/500625

On another note, more or less related but we had a successful upgrade on stable/pike yesterday:
http://logs.openstack.org/45/500145/3/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/9392d26/console.html#_2017-09-04_04_15_27_962324

The main problem now is to make it work on scenarios, which my patch on quickstart might help.
At least it will help to get latest tripleo commits.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/501872

[wes hayutin (weshayutin)]

FYI.. Sagi has pushed a patch to align the yum repo setup.
https://review.openstack.org/#/c/501817/

We should track all the related reviews under a single topic if possible.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/507471

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/507623

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/507801

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/507623
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b4d0a81e55ad51ecdaf2e923f794418ac77cfc57
Submitter: Jenkins
Branch: master

commit b4d0a81e55ad51ecdaf2e923f794418ac77cfc57
Author: Jiri Stransky <email address hidden>
Date: Tue Sep 26 18:31:59 2017 +0200

    Add all services to scenario004-containers

    This commit brings the scenario004 file closer to its BM pendant. We
    need to start with this one to address a chicken-and-egg issue with
    featureset files.

    Change-Id: Ia5c0cefb7051ca42b4d470f5a000eb446d18be30
    Co-Authored-By: Martin André <email address hidden>
    Related-Bug: #1714905
    Related-Bug: #1712070

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/507896

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/507801
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=31550b42027588d82f01db6956c1efaf02d58558
Submitter: Jenkins
Branch: master

commit 31550b42027588d82f01db6956c1efaf02d58558
Author: Jiri Stransky <email address hidden>
Date: Wed Sep 27 11:15:35 2017 +0200

    Stop mapping docker to OS::Heat::None in scenarios

    This was needed to make the upgrade job on Ocata->Pike passing, and we
    now need to remove this to improve the argument order in OOOQ for
    deployments with scenarios.

    This shouldn't be backported to Ocata (at least not before we make the
    split between deploy scenario and upgrade scenario).

    Change-Id: Ie08bbe08530bd48a0ca58667f0704f360e0a4dd7
    Co-Authored-By: Martin André <email address hidden>
    Related-Bug: #1714905
    Related-Bug: #1712070

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/507907

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/507896
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=72e5ec2a136bed7db610be2cb9af7fb510d536f8
Submitter: Jenkins
Branch: stable/pike

commit 72e5ec2a136bed7db610be2cb9af7fb510d536f8
Author: Jiri Stransky <email address hidden>
Date: Tue Sep 26 18:31:59 2017 +0200

    Add all services to scenario004-containers

    This commit brings the scenario004 file closer to its BM pendant. We
    need to start with this one to address a chicken-and-egg issue with
    featureset files.

    Change-Id: Ia5c0cefb7051ca42b4d470f5a000eb446d18be30
    Co-Authored-By: Martin André <email address hidden>
    Related-Bug: #1714905
    Related-Bug: #1712070
    (cherry picked from commit b4d0a81e55ad51ecdaf2e923f794418ac77cfc57)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/507907
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=682686e66b379ab5a04b5cc0752048fb6d2e5100
Submitter: Jenkins
Branch: stable/pike

commit 682686e66b379ab5a04b5cc0752048fb6d2e5100
Author: Jiri Stransky <email address hidden>
Date: Wed Sep 27 11:15:35 2017 +0200

    Stop mapping docker to OS::Heat::None in scenarios

    This was needed to make the upgrade job on Ocata->Pike passing, and we
    now need to remove this to improve the argument order in OOOQ for
    deployments with scenarios.

    This shouldn't be backported to Ocata (at least not before we make the
    split between deploy scenario and upgrade scenario).

    Change-Id: Ie08bbe08530bd48a0ca58667f0704f360e0a4dd7
    Co-Authored-By: Martin André <email address hidden>
    Related-Bug: #1714905
    Related-Bug: #1712070
    (cherry picked from commit 31550b42027588d82f01db6956c1efaf02d58558)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500546
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=fe88bef95b7aae56bc02f5795f58a36c710bbf31
Submitter: Jenkins
Branch: master

commit fe88bef95b7aae56bc02f5795f58a36c710bbf31
Author: Martin André <email address hidden>
Date: Mon Sep 4 14:29:48 2017 +0200

    Use different variables for deploy and upgrade scenarios

    Introduce an upgrade_composable_scenario variable to hold the path to
    the scenario file used in the upgrade command to differentiate with the
    composable_scenario variable for the path of the scenario file used in
    deploy command. Remove the deploy_composable_scenario variable that was
    previously holding a boolean value, which can be inferred from the
    content of the composable_scenario variable.

    Reorder the arguments passed to the deploy command so that the default
    from the docker.yaml environment doesn't overwrite the setting coming
    from the scenario.

    Co-Authored-By: Jiří Stránský <email address hidden>
    Depends-On: Ie08bbe08530bd48a0ca58667f0704f360e0a4dd7
    Change-Id: Iece70e22cd40cb119f2f9eb3c51be7bd4631fade
    Related-Bug: #1714905

[OpenStack Infra (hudson-openstack)]

Change abandoned by Martin André (<email address hidden>) on branch: master
Review: https://review.openstack.org/507471
Reason: Squashed with Icb659509b38575534be27a1881dbe671c40a5436 as they both depend on Ie0e8de54794a9259c0aeb8c67ae0f6a908844093 and there is no good reason to have separate commits.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500552
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=cbeb7b84f064d5b40ea9332d2eff6dd3b8847a8e
Submitter: Jenkins
Branch: master

commit cbeb7b84f064d5b40ea9332d2eff6dd3b8847a8e
Author: Martin André <email address hidden>
Date: Mon Sep 4 14:40:07 2017 +0200

    Use different variables for deploy and upgrade scenarios

    Use the new upgrade_composable_scenario variable from oooq-extra to set
    the composable scenario file used during upgrade.

    Closes-Bug: #1714905

    Change-Id: Ie0e8de54794a9259c0aeb8c67ae0f6a908844093

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501872
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=c504f83c28b986ceb2b92cc0077959158bd11df7
Submitter: Zuul
Branch: master

commit c504f83c28b986ceb2b92cc0077959158bd11df7
Author: Martin André <email address hidden>
Date: Thu Sep 7 22:50:49 2017 +0200

    Add all services to container scenarios

    This commit brings the multinode containers scenario files closer to
    their BM variants to add missing services and turning pacemaker on.
    These require refactorings in OOOQ in order to support non-containerized
    to containerized upgrade jobs across releases. Ceph-ansible is also
    going to be switched separately.

    Co-Authored-By: Jiri Stransky <email address hidden>
    Depends-On: Ie0e8de54794a9259c0aeb8c67ae0f6a908844093
    Change-Id: Icb659509b38575534be27a1881dbe671c40a5436
    Related-Bug: #1714905
    Related-Bug: #1712070

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/512305

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/500671
Reason: indeed, not needed anymore.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/512305
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=c1e7ff180e31f53b198ecc21dd970eee153a9972
Submitter: Zuul
Branch: stable/pike

commit c1e7ff180e31f53b198ecc21dd970eee153a9972
Author: Martin André <email address hidden>
Date: Thu Sep 7 22:50:49 2017 +0200

    Add all services to container scenarios

    This commit brings the multinode containers scenario files closer to
    their BM variants to add missing services and turning pacemaker on.
    These require refactorings in OOOQ in order to support non-containerized
    to containerized upgrade jobs across releases. Ceph-ansible is also
    going to be switched separately.

    Co-Authored-By: Jiri Stransky <email address hidden>
    Depends-On: Ie0e8de54794a9259c0aeb8c67ae0f6a908844093
    Change-Id: Icb659509b38575534be27a1881dbe671c40a5436
    Related-Bug: #1714905
    Related-Bug: #1712070
    (cherry picked from commit c504f83c28b986ceb2b92cc0077959158bd11df7)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-quickstart 2.1.1 release.