Partition modify sometimes reverts to original size

Bug #1999586 reported by Al Bailey
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Al Bailey

Bug Description

Brief Description
-----------------
Sometimes after a partition is created and immediately resized, the new size is not applied

Severity
--------
Minor

Steps to Reproduce
------------------
Create a partition of size 1G
Resize it to be the remainder of the disk
Monitor the resize operation

Expected Behavior
------------------
It should resize

Actual Behavior
----------------
It switches to attempt to resize to the original size

Reproducibility
---------------
Seen using automation tools approximately 1 out of 10 test runs.

System Configuration
--------------------
Duplex

Branch/Pull Time/Commit
-----------------------
Oct 3 2022

Last Pass
---------
Unknown

Timestamp/Logs
--------------
There are no errors
automation logs show an attempt to increase, and the show reverts to the original size

[2022-09-30 17:03:26] host-disk-partition-modify -s 356 compute-1 ed51d027-3b4d-4cfb-a1a3-d0820582ca2d

+-------------+-----------------------------------------------------------------------+
| Property | Value |
+-------------+-----------------------------------------------------------------------+
| device_path | /dev/disk/by-path/pci-0000:08:00.0-sas-0x5001e6757035d000-lun-0-part5 |
| device_node | /dev/sda5 |
| type_guid | ba5eba11-0000-1111-2222-000000000001 |
| type_name | LVM Physical Volume |
| start_mib | 91158 |
| end_mib | 92182 |
| size_mib | 364544 |
| uuid | ed51d027-3b4d-4cfb-a1a3-d0820582ca2d |
| ihost_uuid | 7679e1cd-3884-4105-917b-97a427d1377b |
| idisk_uuid | 74b37ef6-07fa-490e-925a-1d7e4e06dd96 |
| ipv_uuid | None |
| status | Modifying |
| created_at | 2022-09-30T17:03:01.766548+00:00 |
| updated_at | 2022-09-30T17:03:18.811216+00:00 |
+-------------+-----------------------------------------------------------------------+

[2022-09-30 17:03:27] system host-disk-partition-show compute-1 ed51d027-3b4d-4cfb-a1a3-d0820582ca2
+-------------+-----------------------------------------------------------------------+
| Property | Value |
+-------------+-----------------------------------------------------------------------+
| device_path | /dev/disk/by-path/pci-0000:08:00.0-sas-0x5001e6757035d000-lun-0-part5 |
| device_node | /dev/sda5 |
| type_guid | ba5eba11-0000-1111-2222-000000000001 |
| type_name | LVM Physical Volume |
| start_mib | 91158 |
| end_mib | 92182 |
| size_mib | 1024 |
| uuid | ed51d027-3b4d-4cfb-a1a3-d0820582ca2d |
| ihost_uuid | 7679e1cd-3884-4105-917b-97a427d1377b |
| idisk_uuid | 74b37ef6-07fa-490e-925a-1d7e4e06dd96 |
| ipv_uuid | None |
| status | Modifying |
| created_at | 2022-09-30T17:03:01.766548+00:00 |
| updated_at | 2022-09-30T17:03:26.891912+00:00 |
+-------------+-----------------------------------------------------------------------+
Test Activity
-------------
Automated Testing

Workaround
----------
resize the partition again

Revision history for this message
Al Bailey (albailey1974) wrote :

The logs show a sequence of INFO logs: "PART update" at the time of these reverts.
There should only ever be one of those for a partition change, and those logs should only occur when upgrading from centos to debian

By instrumenting the code, it turns out that one structure has integers and another has strings. so visually the values are the same, but the equality check was failing and it was causing a previous DB query of data from the agent for the size to be written back to the DB,

https://review.opendev.org/c/starlingx/config/+/867567

Changed in starlingx:
assignee: nobody → Al Bailey (albailey1974)
Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to config (master)

Reviewed: https://review.opendev.org/c/starlingx/config/+/867567
Committed: https://opendev.org/starlingx/config/commit/8b23b33d2ca127fcd43fef019ccdbcc500ff0d08
Submitter: "Zuul (22348)"
Branch: master

commit 8b23b33d2ca127fcd43fef019ccdbcc500ff0d08
Author: Al Bailey <email address hidden>
Date: Tue Dec 13 20:01:23 2022 +0000

    Reduce the number of partition DB updates

    When the partition code receives a partition structure
    from sysinv-agent, the fields are strings.
    The database fields are integers, so the comparison
    always considers the values to be mis-matched and attempts
    to update the database.

    This can cause stale 'size' values from the agent query
    to overwrite the DB during a partition resize operation.

    Test Plan:
      PASS: create and update several partitions
      PASS: verify that the 'PART update' log is not created
    once for every partition when doing a partition update.

    Closes-Bug: #1999586
    Signed-off-by: Al Bailey <email address hidden>
    Change-Id: I3113e60a09951aa0efc5a6e549b1614976f75f35

Changed in starlingx:
status: In Progress → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.8.0 stx.config stx.storage
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.