Multiple physical drives for nova-local are not supported, resulting in compute node failing to unlock

Bug #2061526 reported by Erickson Silva de Oliveira
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
Erickson Silva de Oliveira

Bug Description

Brief Description
-----------------
The procedure is to add nova-local ephemeral storage to the compute nodes. For example:

   system host-lvg-add $NODE nova-local
   system host-disk-list $NODE | awk '/\/dev\/nvme1n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}
If more than a single physical disk is added, such as calling:

   system host-lvg-add $NODE nova-local
   system host-disk-list $NODE | awk '/\/dev\/nvme1n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}
   system host-disk-list $NODE | awk '/\/dev\/nvme2n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}
   system host-disk-list $NODE | awk '/\/dev\/nvme3n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}
The compute node will fail on unlock with and error similar to:

2024-04-04T21:09:53.310 ^[[1;31mError: 2024-04-04 21:09:53 +0000 Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Exec[vgchange -an nova-local] is already declared at (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 6); cannot redeclare (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 6) (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 6, column: 5) (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 58) on node worker-05

Severity
--------
Provide the severity of the defect.
Major: System/Feature is usable but degraded to a single physical disk.

Steps to Reproduce
------------------
Add two or more physical drives to nova-local on a compute node

   system host-label-assign $NODE openstack-compute-node=enabled avs=enabled

   system host-lvg-add $NODE nova-local
   system host-disk-list $NODE | awk '/\/dev\/nvme1n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}
   system host-disk-list $NODE | awk '/\/dev\/nvme2n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}
   system host-disk-list $NODE | awk '/\/dev\/nvme3n1/{print $2}' | xargs -i system host-pv-add $NODE nova-local {}

   system host-unlock $NODE
Note, this was tested on a compute node with standard controllers.

Expected Behavior
------------------
nova-local created successfully

Actual Behavior
----------------
Failed to create nova-local

Reproducibility
---------------
Reproducible

System Configuration
--------------------
Standard system

Timestamp/Logs
--------------
2024-04-04T21:09:53.310 ^[[1;31mError: 2024-04-04 21:09:53 +0000 Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Exec[vgchange -an nova-local] is already declared at (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 6); cannot redeclare (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 6) (s:ile: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 6, column: 5) (file: /usr/share/puppet/modules/platform/manifests/worker.pp, line: 58) on node worker-05

Workaround
----------
One may add one physical disk to nova-local per lock/unlock. Meaning if one wishes to add three physical disks to nova-local, one can do so by locking, adding a disk, unlock. Then wait for it to be online, lock again and add a second driver.

Changed in starlingx:
status: New → In Progress
tags: added: stx.storage
Changed in starlingx:
assignee: nobody → Erickson Silva de Oliveira (esilvade)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to stx-puppet (master)

Reviewed: https://review.opendev.org/c/starlingx/stx-puppet/+/915438
Committed: https://opendev.org/starlingx/stx-puppet/commit/349f4e97999bc9e7561c0a6bbc775179343f8452
Submitter: "Zuul (22348)"
Branch: master

commit 349f4e97999bc9e7561c0a6bbc775179343f8452
Author: Hediberto C Silva <email address hidden>
Date: Wed Apr 10 14:18:25 2024 -0300

    Fix puppet class to wipe new PV

    With the change made in [1], when processing more than one
    nova-local, puppet fails with "Duplicate declaration", due
    to the exec "vgchange -an nova-local".

    To resolve this, the variable $name was added, so that it
    becomes dynamic.

    Furthermore, the review mentioned above deactivate VG to perform
    wiping, however, it is not activated again after that. So to
    resolve this, another exec was added, so that VG is activated
    after wipe new PG.

    [1]: https://review.opendev.org/c/starlingx/stx-puppet/+/863871

    Test Plan:
    SX: Delete instances fs and add 4 nova-local
         B&R with nova-local instead of instances

    DX+: Add 4 nova-local in compute-0
         B&R with nova-local created

    STD: Add 4 nova-local in compute-0
         B&R with nova-local created

    Closes-Bug: 2061526

    Change-Id: I7449c5cd7199541551dccee17e22a8bda48414e1
    Signed-off-by: Hediberto C Silva <email address hidden>
    Signed-off-by: Erickson Silva de Oliveira <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Medium
tags: added: stx.10.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.