Debian Patch Removal Fails due to incorrect ordering

[Jessica Castelino]

Brief Description

After installing and removing NRR_INSVC test patch with no problems, while installing and removing RR_ALL_NODES and RR_ALL_NODES_REQUIRES test patches, controller-0 remains not patch current and stuck in locked state as It can't be unlocked due to patch state.

The issue is exposed by the order of these logs
2022-12-17T20:36:00: sw-patch-controller-daemon[4833]: patch_controller.py(1321): INFO: Removing patch: RR_ALL_NODES
2022-12-17T20:36:08: sw-patch-controller-daemon[4833]: patch_controller.py(1321): INFO: Removing patch: RR_ALL_NODES_REQUIRES

the problem here is that we ONLY want to update the summary for RR_ALL_NODES since it is what ends up being the final 'head' for ostree.
RR_ALL_NODES_REQUIRES can mostly be ignored since we are skipping right over it by removing both patches.

We have 'requires' setup to prevent the base commit from being removed without the requires also being removed, so now we just need to either re-order the sequence, or determine that the summary update can be skipped

Severity:
-----------------------------------------
Major: controller-0 stuck in locked state

Steps to Reproduce
--------------------------------------------------------
Build 2022-12-15_22-00-07:

1) sudo sw-patch upload RR_ALL_NODES.patch RR_ALL_NODES_REQUIRES.patch

2) sudo sw-patch apply RR_ALL_NODES RR_ALL_NODES_REQUIRES

3) system host-lock controller-0 (after this step host was showed as patch current and RR)

4) sudo sw-patch host-install controller-0

5) system host-unlock controller-0 (after this, patch current and NO RR)

6) sudo sw-patch remove RR_ALL_NODES RR_ALL_NODES_REQUIRES (after this, host RR and not patch current)

7) system host-lock controller-0

8) sudo sw-patch host-install controller-0 (success but host remains not patch current)

9) system host-unlock controller-0
out: host-unlock rejected: Not patch current. 'sw-patch host-install controller-0' is required.

Expected Behavior
-----------------------------
Removing the two patches should be supported, host should be patch current and able to unlock

Actual Behavior
---------------------------------------------
Host is not patch current and unable to unlock

Reproducibility
----------------------------------------------
Reproducible

System Configuration
-----------------------------------------
One node system IPv4

Load info:
---------------------------------------
BUILD_DATE="2022-12-16 03:00:07 +0000"

Last Pass
----------------------------------------
NA

Timestamp/Logs
-------------------------
NA

Alarms
-----------------------------
No alarms at the time TC started

After:

+----------+---------------------------------------------------------------------+-------------------+----------+---------------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+---------------------------------------------------------------------+-------------------+----------+---------------------+
| 200.001 | controller-0 was administratively locked to take it out-of-service. | host=controller-0 | warning | 2022-12-17T17:34:45 |
| | | | | .191831 |
| | | | | |
+----------+---------------------------------------------------------------------+-------------------+----------+---------------------+

Test Activity
---------------------------
Regression Testing

Workaround
---------------------
No workaround known

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/update/+/869855

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/update/+/869855
Committed: https://opendev.org/starlingx/update/commit/2b172c641a00c9c935333bcc944a447299bcbf32
Submitter: "Zuul (22348)"
Branch: master

commit 2b172c641a00c9c935333bcc944a447299bcbf32
Author: Jessica Castelino <email address hidden>
Date: Tue Jan 10 21:28:01 2023 +0000

    Fix: Patch Remove Order

    When we remove a patch, we want the patch removal order to be
    such that the patch with the highest dependency is removed first.
    For example:
    P2 requires P1
    P3 requires P2
    P4 requires P3
    Removal order: [P4, P3, P2, P1]
    Since P4 has the highest dependencies (P3, P2, P1) followed by
    P3 (P2, P1) followed by P2 (P1) and P1 (None).

    Test Plan:
    Tested different scenarios with unit test cases

    Closes-Bug: 2002437
    Signed-off-by: Jessica Castelino <email address hidden>
    Change-Id: I6c7a255fef3934e51b92d3527b3412a13ba9b7c5