upgrade controller-0 failure with Error: /dev/sda: unrecognised disk label

[Heitor Matsui]

Brief Description
-----------------
The Central cloud, the controller-0 upgrade failed based on the logs the error was on disk partition creation.

Severity
--------
Critical

Steps to Reproduce
------------------
1. DC system installed with stx5 with patches
2. DC system upgraded from stx5 to stx6 as per upgrade procedure
3. K8s upgraded
4. Upgrade enabled patch applied
5. Follow the upgrade procedure to upgrade DC system upgraded from stx6 to stx8. During this Central cloud upgrade, The Controller-0 upgrade failed.

Expected Behavior
------------------
Upgrade successful with partition creation

Actual Behavior
----------------
Kickstarts and puppet failures

Reproducibility
---------------
N/A

System Configuration
--------------------
DC

Branch/Pull Time/Commit
-----------------------
2023-03-16 22:47:37

Last Pass
---------
N/A

Timestamp/Logs
--------------
2023-03-22T12:21:19.831 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: sysinv 2023-03-22 12:21:19.641 77121 CRITICAL sysinv [-] Unhandled error: OSError: Could not create partition 2 of 1024MiB on disk /dev/disk/by-path/pci-0000:05:00.0-scsi-0:0:12:0: Error: You requested a partition from 1.00MiB to 1025MiB (sectors 2048..2099199).
2023-03-22T12:21:19.832 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: The closest location we can manage is 1.00MiB to 1.00MiB (sectors 2047..2047).
2023-03-22T12:21:19.834 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv Traceback (most recent call last):
2023-03-22T12:21:19.836 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv File "/usr/bin/manage-partitions", line 903, in <module>
2023-03-22T12:21:19.838 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv main(sys.argv)
2023-03-22T12:21:19.840 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv File "/usr/bin/manage-partitions", line 895, in main
2023-03-22T12:21:19.842 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv run(CONF.action.func, CONF.action.data,
2023-03-22T12:21:19.844 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv File "/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py", line 360, in inner
2023-03-22T12:21:19.846 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv return f(*args, **kwargs)
2023-03-22T12:21:19.848 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv File "/usr/bin/manage-partitions", line 878, in run
2023-03-22T12:21:19.850 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv action(data, mode, pfile)
2023-03-22T12:21:19.852 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv File "/usr/bin/manage-partitions", line 853, in check_partitions
2023-03-22T12:21:19.854 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv _create_partition(disk, partition_number, p.get('start_mib'),
2023-03-22T12:21:19.856 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv File "/usr/bin/manage-partitions", line 327, in _create_partition
2023-03-22T12:21:19.858 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv raise IOError("Could not create partition %s of %sMiB on disk %s: %s" %
2023-03-22T12:21:19.860 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv OSError: Could not create partition 2 of 1024MiB on disk /dev/disk/by-path/pci-0000:05:00.0-scsi-0:0:12:0: Error: You requested a partition from 1.00MiB to 1025MiB (sectors 2048..2099199).
2023-03-22T12:21:19.862 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv The closest location we can manage is 1.00MiB to 1.00MiB (sectors 2047..2047).
2023-03-22T12:21:19.864 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv
2023-03-22T12:21:19.865 Notice: 2023-03-22 12:21:19 +0000 /Stage[main]/Platform::Partitions/Platform_manage_partition[check]/Exec[manage-partitions-check]/returns: 2023-03-22 12:21:19.641 77121 ERROR sysinv

Test Activity
-------------
Upgrade Testing

Workaround
----------
N/A

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/878801
Committed: https://opendev.org/starlingx/config/commit/4140ac7e510b117e5ca55d09e22c1db4a6479670
Submitter: "Zuul (22348)"
Branch: master

commit 4140ac7e510b117e5ca55d09e22c1db4a6479670
Author: Heitor Matsui <email address hidden>
Date: Tue Mar 28 16:04:54 2023 -0300

    Skip ceph partitions on manage-partitions script

    There are some scenarios where manage-partitions script
    identifies missing ceph partitions on the ceph disk and
    tries to create them, leading to errors.

    This commit skips ceph partitions on manage-partitions
    script, since ceph partitions are special and should be
    managed only by the specific ceph puppet manifests.

    Test Plan:
    PASS: AIO-DX/SX and Standard upgrade
    PASS: AIO-DX/SX and Standard install/bootstrap/unlock
    PASS: AIO-DX/SX and Standard host-lock/unlock and verify
          manage-partitions log skipping ceph partitions

    Closes-bug: 2013380

    Signed-off-by: Heitor Matsui <email address hidden>
    Change-Id: I4f377a664479daf4e0f13625580096a843ff32ff