Lifecycle converge commands don't remove no-ops

[Jiří Stránský]

( also discussed in https://bugzilla.redhat.com/show_bug.cgi?id=1573922 )

Removing no-ops from plan only will have no effect, we need to let
Heat know about the no-ops removal too. Client commands always fully
recreate the plan in swift, so any plan edits need to be immediately
applied.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/565828

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/566006

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/566009

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/566043

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/566050

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/566051

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jiri Stransky (<email address hidden>) on branch: master
Review: https://review.openstack.org/566043
Reason: Merged into https://review.openstack.org/#/c/566051/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566050
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=1612b374d7b6969a06351de2bd43d04e8b5e4fb6
Submitter: Zuul
Branch: master

commit 1612b374d7b6969a06351de2bd43d04e8b5e4fb6
Author: Giulio Fidente <email address hidden>
Date: Wed Apr 25 18:03:11 2018 +0200

    Stop trying to converge during Ceph upgrade command

    This had no effect as we only updated the plan and not the
    stack. We'll do a normal converge after Ceph upgrade command, which
    will also take care of putting back the normal deploy
    CephAnsiblePlaybook.

    Furthermore it was prepending the converge env file to a list of envs
    which already had the *prepare* env file at a latter position (we
    reused the env list from the prepare call :) ), so it kept doing
    prepare operation instead of converge.

    And perhaps most importantly, the messages printed to user are
    misleading since no converge was really happening.

    Also adds support for --container-registry-file to ceph-upgrade command,
    while not necessary, some of the functions called by ceph-upgrade
    command expect to receive the container-registry parameters.

    Co-Authored-By: Giulio Fidente <email address hidden>
    Change-Id: I025eac40f8bda5f23c789e7fef1a9e9b49947f66
    Partial-Bug: #1768586

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/566944

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566944
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=5d1450a77432bc0c4b7d0dbb76de7efd7d39c56b
Submitter: Zuul
Branch: stable/queens

commit 5d1450a77432bc0c4b7d0dbb76de7efd7d39c56b
Author: Giulio Fidente <email address hidden>
Date: Wed Apr 25 18:03:11 2018 +0200

    Stop trying to converge during Ceph upgrade command

    This had no effect as we only updated the plan and not the
    stack. We'll do a normal converge after Ceph upgrade command, which
    will also take care of putting back the normal deploy
    CephAnsiblePlaybook.

    Furthermore it was prepending the converge env file to a list of envs
    which already had the *prepare* env file at a latter position (we
    reused the env list from the prepare call :) ), so it kept doing
    prepare operation instead of converge.

    And perhaps most importantly, the messages printed to user are
    misleading since no converge was really happening.

    Also adds support for --container-registry-file to ceph-upgrade command,
    while not necessary, some of the functions called by ceph-upgrade
    command expect to receive the container-registry parameters.

    Co-Authored-By: Giulio Fidente <email address hidden>
    Change-Id: I025eac40f8bda5f23c789e7fef1a9e9b49947f66
    Partial-Bug: #1768586
    (cherry picked from commit 1612b374d7b6969a06351de2bd43d04e8b5e4fb6)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/567243

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566051
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=4742f2ed5a28e5371e52ad0ffa6ed4f280d992bd
Submitter: Zuul
Branch: master

commit 4742f2ed5a28e5371e52ad0ffa6ed4f280d992bd
Author: Jiri Stransky <email address hidden>
Date: Thu May 3 14:08:14 2018 +0200

    Set normal CephAnsiblePlaybook on update/upgrade/ffwd converge

    As we discovered bug #1768586 we'll need to make sure that every
    parameter tweak in plan is followed by a stack update.

    So far the Ceph upgrade command did set param -> stack update -> unset
    param (only in plan). However this means the last CephAnsiblePlaybook
    setting (back to normal deploy playbook) was discarded.

    Let's reuse normal converge commands to converge CephAnsiblePlaybook
    too, and we can remove the (now unused) ceph-upgrade-converge.yaml. We
    won't do more stack updates than necessary, and at the same time the
    user workflow stays somewhat consistent between envs that do and don't
    have Ceph.

    An alternative would be to run the as part of the Ceph command, but
    that either means we'd have to run one more stack update then
    necessary, or skip the last converge in envs with Ceph, and
    essentially diverge further from the non-Ceph workflow.

    Change-Id: If596531cbb1e750ed67e66391743f4c1833e4337
    Depends-On: I025eac40f8bda5f23c789e7fef1a9e9b49947f66
    Partial-Bug: #1768586

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/567275

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566006
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=6e30581a7255ca13e27cd8d87cb09fc1b6fdfd31
Submitter: Zuul
Branch: master

commit 6e30581a7255ca13e27cd8d87cb09fc1b6fdfd31
Author: Jiri Stransky <email address hidden>
Date: Thu May 3 11:35:30 2018 +0200

    Don't run full update/upgrade converge in CI to keep sane wall time

    We discovered we need to put back stack updates into the converge
    operations. This increases the run time of the upgrade procedure
    considerably, and e.g. our voting CI job for updates might go over the
    time limit. We'll need to start passing --update-plan-only to the
    converge operation to keep CI on previous run times.

    Change-Id: Idc5dbebc116aba2420d275aab3981186511f298e
    Related-Bug: #1768586

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566009
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=dea69896c441d0d562513d93b60cb6b2e18c22f7
Submitter: Zuul
Branch: master

commit dea69896c441d0d562513d93b60cb6b2e18c22f7
Author: Jiri Stransky <email address hidden>
Date: Thu May 3 11:38:13 2018 +0200

    Allow running update/upgrade converge with --update-plan-only

    We discovered we need to put back stack updates into the converge
    operations. This increases the run time of the upgrade procedure
    considerably, and e.g. our voting CI job for updates might go over the
    time limit. We'll need to start passing --update-plan-only to the
    converge operation to keep CI on previous run times.

    Depends-On: Idc5dbebc116aba2420d275aab3981186511f298e
    Change-Id: I9039db99f43e8ad091ea6fef7e3a149fcddf1334
    Related-Bug: #1768586

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565828
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=a0006df5442ee94637f236c46cf1a0fb436e5c0a
Submitter: Zuul
Branch: master

commit a0006df5442ee94637f236c46cf1a0fb436e5c0a
Author: Jiri Stransky <email address hidden>
Date: Wed May 2 17:58:58 2018 +0200

    Update Heat stack on update/upgrade/ffwd converge

    Removing no-ops from plan only will have no effect, we need to let
    Heat know about the no-ops removal too. Client commands always fully
    recreate the plan in swift, so any plan edits need to be immediately
    applied.

    For the same reason this commit also drops execution of the workfow --
    so far we thought it's just superfluous (does what the env file did),
    but in fact can't have any effect at all, so let's not execute that
    code at all.

    Depends-On: I9039db99f43e8ad091ea6fef7e3a149fcddf1334
    Change-Id: Ife57bfae034f70e33bb64cfc3f3063b6a5528cc6
    Closes-Bug: #1768586

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alex Schultz (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/567243
Reason: abandoning to clear gate for Bug 1770355. Will restore after fix lands.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/568222

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/568222
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=468257d2f403ad63e19c6339ef7617928d1ae8e6
Submitter: Zuul
Branch: stable/queens

commit 468257d2f403ad63e19c6339ef7617928d1ae8e6
Author: Jiri Stransky <email address hidden>
Date: Thu May 3 11:38:13 2018 +0200

    Allow running update/upgrade converge with --update-plan-only

    We discovered we need to put back stack updates into the converge
    operations. This increases the run time of the upgrade procedure
    considerably, and e.g. our voting CI job for updates might go over the
    time limit. We'll need to start passing --update-plan-only to the
    converge operation to keep CI on previous run times.

    Depends-On: Idc5dbebc116aba2420d275aab3981186511f298e
    Change-Id: I9039db99f43e8ad091ea6fef7e3a149fcddf1334
    Related-Bug: #1768586
    (cherry picked from commit dea69896c441d0d562513d93b60cb6b2e18c22f7)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 10.1.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/567275
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=8c4a9a8109a005e91f32858b172d853bf6525fce
Submitter: Zuul
Branch: stable/queens

commit 8c4a9a8109a005e91f32858b172d853bf6525fce
Author: Jiri Stransky <email address hidden>
Date: Thu May 3 14:08:14 2018 +0200

    Set normal CephAnsiblePlaybook on update/upgrade/ffwd converge

    As we discovered bug #1768586 we'll need to make sure that every
    parameter tweak in plan is followed by a stack update.

    So far the Ceph upgrade command did set param -> stack update -> unset
    param (only in plan). However this means the last CephAnsiblePlaybook
    setting (back to normal deploy playbook) was discarded.

    Let's reuse normal converge commands to converge CephAnsiblePlaybook
    too, and we can remove the (now unused) ceph-upgrade-converge.yaml. We
    won't do more stack updates than necessary, and at the same time the
    user workflow stays somewhat consistent between envs that do and don't
    have Ceph.

    An alternative would be to run the as part of the Ceph command, but
    that either means we'd have to run one more stack update then
    necessary, or skip the last converge in envs with Ceph, and
    essentially diverge further from the non-Ceph workflow.

    Change-Id: If596531cbb1e750ed67e66391743f4c1833e4337
    Depends-On: I025eac40f8bda5f23c789e7fef1a9e9b49947f66
    Partial-Bug: #1768586
    (cherry picked from commit 4742f2ed5a28e5371e52ad0ffa6ed4f280d992bd)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/567243
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=1dec392c4c48ac6d2f5e4a3526219c5c94dec3ef
Submitter: Zuul
Branch: stable/queens

commit 1dec392c4c48ac6d2f5e4a3526219c5c94dec3ef
Author: Jiri Stransky <email address hidden>
Date: Wed May 2 17:58:58 2018 +0200

    Update Heat stack on update/upgrade/ffwd converge

    Removing no-ops from plan only will have no effect, we need to let
    Heat know about the no-ops removal too. Client commands always fully
    recreate the plan in swift, so any plan edits need to be immediately
    applied.

    For the same reason this commit also drops execution of the workfow --
    so far we thought it's just superfluous (does what the env file did),
    but in fact can't have any effect at all, so let's not execute that
    code at all.

    Depends-On: I9039db99f43e8ad091ea6fef7e3a149fcddf1334
    Change-Id: Ife57bfae034f70e33bb64cfc3f3063b6a5528cc6
    Closes-Bug: #1768586
    (cherry picked from commit a0006df5442ee94637f236c46cf1a0fb436e5c0a)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 9.2.3 release.