System Allows Upgrade while a restore is not marked complete

[Dipankar Kar]

Brief Description

If an upgrade initiated without marking the restore complete, then the upgrade will fail because sysinv will skip downloading images in a restore setting.

Customer facing Backup & Restore documentation does indicate that the restore should be marked complete, so no documentation updates are required.

Severity

Major - An upgrade can fail if the operator fails to mark the restore as complete.

Steps to Reproduce

Install centos version of stx
Upgrade to any other later version.
Restore the lab.
Do not mark the restore completed.
Upgrade controllers
Run activation-upgrade
At this point there will be a failure.
Expected Behavior

There should be a Symantec check that the restore is marked complete in two locations:

The output of "system health-query-upgrade" should show a FAIL if a restore is in progress.
Running "system upgrade-upgrade <controller>" should not run in the event a restore is in progress.

Actual Behavior

The system allows the upgrade to proceed, during the upgrade sysinv starts and skips downloading required images, and then upgrade-activate step fails.

Reproducibility

100%

Last Pass

This scenario has not been identified before.

Workaround

Make sure to run "system restore-show" and do not proceed with any upgrade step unless the output is "Restore procedure not in progress"

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/906271

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/906271
Committed: https://opendev.org/starlingx/config/commit/19cbe5b4903de647666c276c7c3e9512439d9a97
Submitter: "Zuul (22348)"
Branch: master

commit 19cbe5b4903de647666c276c7c3e9512439d9a97
Author: Dipankar Kar <email address hidden>
Date: Mon Jan 22 04:48:50 2024 -0500

    Add semantic check for Restore in progress

    There is no semantic check for restore in progress
    during upgrade, if the system is not simplex then
    upgrade activation step will fail.

    This semantic check will reject upgrade activation
    and force to restart the upgrade again for non simplex
    systems.

    TEST PLAN:
    PASS: AIO-SX: Local backup and Restore
    PASS: AIO-DX: Upgrade 21.12 to 22.12-prestaging
    PASS: AIO-DX: Local backup and Restore
    PASS: DCManager subcloud backup

    Closes-bug: 2050096

    Change-Id: I6d2e4f6e406a52ca24b570ed2f634389d1b9083a
    Signed-off-by: Dipankar Kar <email address hidden>