Compute services not getting deleted after scale down

[David Vallee Delisle]

When we scaledown, we normally source the undercloud's credentials (stackrc).

The openstack command that calls for the compute service deletion is simply setting the environment variable OS_CLOUD to the overcloud's stack name (or RootStackName), but this is not enough to hit the overcloud's nova API. Scale down fails when running on novaless deployment because of that.

(undercloud) [stack@undercloud-0 ~]$ export OS_CLOUD=overcloud
(undercloud) [stack@undercloud-0 ~]$ openstack compute service list
+----+----------------+---------------------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+----------------+---------------------------+----------+---------+-------+----------------------------+
| 1 | nova-conductor | undercloud-0.redhat.local | internal | enabled | up | 2021-06-07T22:03:28.000000 |
| 2 | nova-scheduler | undercloud-0.redhat.local | internal | enabled | up | 2021-06-07T22:03:29.000000 |
| 6 | nova-compute | undercloud-0.redhat.local | nova | enabled | up | 2021-06-07T22:03:24.000000 |
+----+----------------+---------------------------+----------+---------+-------+----------------------------+
(undercloud) [stack@undercloud-0 ~]$ for key in $( set | awk -F= '/^OS_/ {print $1}' ); do unset "${key}" ; done
 [stack@undercloud-0 ~]$ openstack compute service list
Missing value auth-url required for auth plugin password
 [stack@undercloud-0 ~]$ export OS_CLOUD=overcloud
 [stack@undercloud-0 ~]$ openstack compute service list
+----+----------------+---------------------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+----------------+---------------------------+----------+---------+-------+----------------------------+
| 1 | nova-conductor | controller-0.redhat.local | internal | enabled | up | 2021-06-07T22:04:10.000000 |
| 3 | nova-scheduler | controller-0.redhat.local | internal | enabled | up | 2021-06-07T22:04:03.000000 |
| 6 | nova-compute | compute-0.redhat.local | nova | enabled | up | 2021-06-07T22:04:10.000000 |
+----+----------------+---------------------------+----------+---------+-------+----------------------------+

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/795198

[Brendan Shephard (bshephar)]

@David. Are you able to test with the proposed patch?

I tested this in a older oooq lab I have running and it seems to get passed the issue. Note that this issue is related to the change made here:
https://bugs.launchpad.net/tripleo/+bug/1913275
https://review.opendev.org/c/openstack/tripleo-heat-templates/+/776328

[Brendan Shephard (bshephar)]

This is probably a cleaner solution:
https://review.opendev.org/c/openstack/tripleo-heat-templates/+/795203

I'll test this as well.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-common/+/795225

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Brendan Shephard <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/795198
Reason: Better solution proposed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/795225
Committed: https://opendev.org/openstack/tripleo-common/commit/1371815d554d6ad85a661f73f211579d9348752b
Submitter: "Zuul (22348)"
Branch: master

commit 1371815d554d6ad85a661f73f211579d9348752b
Author: ramishra <email address hidden>
Date: Tue Jun 8 12:23:09 2021 +0530

    Cleanup OS_CLOUD env var for ephemeral heat

    This would cleanup OS_CLOUD if already set. Otherwise there
    would be conflicts.

    Related-Bug: #1931172
    Change-Id: Idcbe3f248ce631310245e35214b51b38263956b6

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/795203
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/454b45c7d2705f75bb06c5e60470e5aa0264923a
Submitter: "Zuul (22348)"
Branch: master

commit 454b45c7d2705f75bb06c5e60470e5aa0264923a
Author: ramishra <email address hidden>
Date: Tue Jun 8 08:33:35 2021 +0530

    Set OS_CLOUD instead in stackrc

    We now write clouds.yaml when configuring keystone and
    can use OS_CLOUD instead. Setting OS_AUTH_URL overrides
    when we want use OS_CLOUD pointing to overcloud.

    Closes-bug: #1931172
    Depends-On: https://review.opendev.org/c/openstack/tripleo-common/+/795225
    Depends-On: https://review.opendev.org/c/openstack/tripleo-validations/+/795293
    Change-Id: Ib42fc397389d8b50c740edf334fe6f281c392ec6

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-common/+/797207

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/797209

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/797207
Committed: https://opendev.org/openstack/tripleo-common/commit/d2552f6e646a5282b5cdff161ecc05c80428fcb4
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit d2552f6e646a5282b5cdff161ecc05c80428fcb4
Author: ramishra <email address hidden>
Date: Tue Jun 8 12:23:09 2021 +0530

    Cleanup OS_CLOUD env var for ephemeral heat

    This would cleanup OS_CLOUD if already set. Otherwise there
    would be conflicts.

    Related-Bug: #1931172
    Change-Id: Idcbe3f248ce631310245e35214b51b38263956b6
    (cherry picked from commit 1371815d554d6ad85a661f73f211579d9348752b)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/797209
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/e2ff0b44649ece5257a84eca76c5b33a1a0fd88a
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit e2ff0b44649ece5257a84eca76c5b33a1a0fd88a
Author: ramishra <email address hidden>
Date: Tue Jun 8 08:33:35 2021 +0530

    Set OS_CLOUD instead in stackrc

    We now write clouds.yaml when configuring keystone and
    can use OS_CLOUD instead. Setting OS_AUTH_URL overrides
    when we want use OS_CLOUD pointing to overcloud.

    Closes-bug: #1931172
    Depends-On: https://review.opendev.org/c/openstack/tripleo-common/+/797207
    Depends-On: https://review.opendev.org/c/openstack/tripleo-validations/+/797208
    Change-Id: Ib42fc397389d8b50c740edf334fe6f281c392ec6
    (cherry picked from commit 454b45c7d2705f75bb06c5e60470e5aa0264923a)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 14.2.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/801888
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/34a009b03fed08de8a9121bc10116a90c7fb5878
Submitter: "Zuul (22348)"
Branch: stable/train

commit 34a009b03fed08de8a9121bc10116a90c7fb5878
Author: David Vallee Delisle <email address hidden>
Date: Wed Jun 23 13:03:52 2021 -0400

    vncproxy: removal of tags and steps change from post_upgrade_tasks

    Post upgrade tasks are limited to 4-1 steps, so these
    tasks are not executed.

    Deleting nova-consoleauth is failing with some of the CIs and this is
    mostly a best-effort kind of task, so adding ignore_errors.

    Also, replacing some yes/nos with boolean as it's best practice.

    NOTE(dvd): train-only modification: reverting back OS_CLOUD change as it
    conflicts with the sourced stackrc credentials when running a scale down
    as described in this LP#1931172

    Related: https://bugzilla.redhat.com/1921112
    Related-bug: #1931172
    Change-Id: I0bc39170569c42a0be31233fd7ec068613547790
    (cherry picked from commit 2001b48ca3430746f407b69d31d9e7e19303f9b8)
    (cherry picked from commit 28c866c3e9acde7c34216e059d79ba96a43e6ddb)
    (cherry picked from commit e5dbf568a60616ddf22b645183854f97d807f0b7)
    (cherry picked from commit 76aa869fa6f1326bba71b932707c51426e415cf6)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 15.1.0 release.