missing config option to cleanup instance dir of evacuated instances

Bug #2019141 reported by Rodrigo Barbieri
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Charm Guide
Fix Released
High
Rodrigo Barbieri
OpenStack Nova Compute Charm
Fix Committed
High
Rodrigo Barbieri
2023.1
Fix Released
Undecided
Unassigned
Ussuri
Fix Committed
Undecided
Unassigned
Victoria
Fix Committed
Undecided
Unassigned
Wallaby
Fix Committed
Undecided
Unassigned
Xena
Fix Released
Undecided
Unassigned
Yoga
Fix Released
Undecided
Unassigned
Zed
Fix Released
Undecided
Unassigned

Bug Description

Need to be able to configure ensure_libvirt_rbd_instance_dir_cleanup to address:

https://bugs.launchpad.net/nova/+bug/1414895

https://bugs.launchpad.net/nova/+bug/1761062

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (master)
Changed in charm-nova-compute:
status: New → In Progress
Revision history for this message
Felipe Reyes (freyes) wrote :

We had a discussion with Rodrigo, Dan and Alex where we agreed that the path forward to fix this issue is the following:

Check if nova-compute is related to ceph-mon and configured with libvirt-image-backend=rbd [0], then [workarounds].ensure_libvirt_rbd_instance_dir_cleanup is set to True, unless instances-path[1] is set to a non-default value where the charm will go into blocked state informing the user about the unsupported configuration.

We agreed this will improve the situation for operators using Charmed OpenStack while still putting safeguards to minimize the risk of losing data.

[0] https://charmhub.io/nova-compute/configure#libvirt-image-backend
[1] https://charmhub.io/nova-compute/configure#instances-path

Changed in charm-nova-compute:
assignee: nobody → Rodrigo Barbieri (rodrigo-barbieri2010)
importance: Undecided → High
Revision history for this message
Felipe Reyes (freyes) wrote :

Hi Rodrigo, I just added a task for the charm-guide, so this bug fix receives a new entry in the release notes. Thanks

Changed in charm-guide:
assignee: nobody → Rodrigo Barbieri (rodrigo-barbieri2010)
status: New → Triaged
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/2023.1)

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/890659

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/2023.1)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/890659
Committed: https://opendev.org/openstack/charm-nova-compute/commit/003f6b6d70fe7998c57095abb6ac97d97cd2dff1
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)

Felipe Reyes (freyes)
Changed in charm-nova-compute:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/zed)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/892334

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/zed)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/892299
Committed: https://opendev.org/openstack/charm-nova-compute/commit/f6a2da6d187d6fe53195e7b123e845e2f80ebb4d
Submitter: "Zuul (22348)"
Branch: stable/zed

commit f6a2da6d187d6fe53195e7b123e845e2f80ebb4d
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)
    (cherry picked from commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1)

tags: added: in-stable-zed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/892334
Committed: https://opendev.org/openstack/charm-nova-compute/commit/1ddfe5e6c212804ac3623dc4782fa7e51da840a0
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 1ddfe5e6c212804ac3623dc4782fa7e51da840a0
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)
    (cherry picked from commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1)
    (cherry picked from commit cfafa8cf5f2d880635e66e9620ca9d0334375942)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/897322

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/897323

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/897324

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-nova-compute (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/897325

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/897322
Committed: https://opendev.org/openstack/charm-nova-compute/commit/d3e4cc31c63e157efdc3e548a44c3f4c8f2bcdcd
Submitter: "Zuul (22348)"
Branch: stable/xena

commit d3e4cc31c63e157efdc3e548a44c3f4c8f2bcdcd
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)
    (cherry picked from commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1)
    (cherry picked from commit cfafa8cf5f2d880635e66e9620ca9d0334375942)
    (cherry picked from commit 1ddfe5e6c212804ac3623dc4782fa7e51da840a0)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/897323
Committed: https://opendev.org/openstack/charm-nova-compute/commit/be845826f95f812370b33fbdd22b6fd70ebb217f
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit be845826f95f812370b33fbdd22b6fd70ebb217f
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)
    (cherry picked from commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1)
    (cherry picked from commit cfafa8cf5f2d880635e66e9620ca9d0334375942)
    (cherry picked from commit 1ddfe5e6c212804ac3623dc4782fa7e51da840a0)
    (cherry picked from commit d3e4cc31c63e157efdc3e548a44c3f4c8f2bcdcd)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/897324
Committed: https://opendev.org/openstack/charm-nova-compute/commit/f1ff0d55b8ebcb0e3b6417d9fcd08da334df52cf
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit f1ff0d55b8ebcb0e3b6417d9fcd08da334df52cf
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)
    (cherry picked from commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1)
    (cherry picked from commit cfafa8cf5f2d880635e66e9620ca9d0334375942)
    (cherry picked from commit 1ddfe5e6c212804ac3623dc4782fa7e51da840a0)
    (cherry picked from commit d3e4cc31c63e157efdc3e548a44c3f4c8f2bcdcd)
    (cherry picked from commit be845826f95f812370b33fbdd22b6fd70ebb217f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/897325
Committed: https://opendev.org/openstack/charm-nova-compute/commit/db6fad9810388f7edf1aa2725bb30f82137dd063
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit db6fad9810388f7edf1aa2725bb30f82137dd063
Author: Rodrigo Barbieri <email address hidden>
Date: Wed May 10 13:33:08 2023 -0300

    Set nova config for rbd instance folder cleanup after evacuations

    After evacuations and revert resizes when using rbd storage backend,
    the instance folder is usually left behind and causes issues when
    migrating the instance back to the host.
    With the config option set, the nova-compute service will cleanup
    those folders as part of the periodic checks that run for instances
    that have been evacuated/migrated.

    Closes-bug: #2019141
    Change-Id: I846ccb0a95d04139b41fdad6cbf465d303d6cc09
    (cherry picked from commit e61d89aa47cba71bb4dda12d836fde8a8fa7092c)
    (cherry picked from commit 003f6b6d70fe7998c57095abb6ac97d97cd2dff1)
    (cherry picked from commit cfafa8cf5f2d880635e66e9620ca9d0334375942)
    (cherry picked from commit 1ddfe5e6c212804ac3623dc4782fa7e51da840a0)
    (cherry picked from commit d3e4cc31c63e157efdc3e548a44c3f4c8f2bcdcd)
    (cherry picked from commit be845826f95f812370b33fbdd22b6fd70ebb217f)
    (cherry picked from commit f1ff0d55b8ebcb0e3b6417d9fcd08da334df52cf)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to charm-guide (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-guide/+/900405

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to charm-guide (master)

Reviewed: https://review.opendev.org/c/openstack/charm-guide/+/900405
Committed: https://opendev.org/openstack/charm-guide/commit/271d39c91d480a669a5a46f303932a4d8ccbb1c7
Submitter: "Zuul (22348)"
Branch: master

commit 271d39c91d480a669a5a46f303932a4d8ccbb1c7
Author: Alex Kavanagh <email address hidden>
Date: Wed Nov 8 12:25:10 2023 +0200

    Add release note for bug: 2019141

    Add note explaining impact of change of 2019141 on the operation of the
    nova-compute service with rbd storage.

    Change-Id: I9319c80af0159d28a5ef845216ee0aed5d78f33d
    Related-Bug: #2019141

Changed in charm-guide:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.