Volume marked as available after a failure to build

Bug #1899649 reported by Lee Yarwood
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Lee Yarwood
Victoria
Fix Released
Medium
Lee Yarwood

Bug Description

Description
===========
When skipping the scheduler by specifying a host in the provided availability_zone parameter of a request ('$az:$host') any failure to build an instance with volumes attached on the target host will result in the removal of all volume attachments within Cinder. This in turn marks this volume as available, allowing future requests to potentially use the volume.

This is due to the following exception handling code being used when no retry info is provided when skipping the scheduler:

https://github.com/openstack/nova/blob/261de76104ca67bed3ea6cdbcaaab0e44030f1e2/nova/compute/manager.py#L2225-L2241

This is contrast to what happens after a failure to schedule an instance with volumes attached where the volume attachments remain and the volume is left in a reserved state.

Steps to reproduce
==================
* Attempt to spawn an instance on a specific host using the availability_zone parameter.
* Ensure this request fails during _build_and_run_instance raising exception.RescheduledException

Expected result
===============
Any associated volumes still have attachments to the instance and are left in a reserved state.

Actual result
=============
All volume attachments are removed and any associated volumes are left in an available state.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/

   1e10461c71cb78226824988b8c903448ba7a8a76

2. Which hypervisor did you use?
   (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
   What's the version of that?

   N/A

2. Which storage type did you use?
   (For example: Ceph, LVM, GPFS, ...)
   What's the version of that?

   N/A

3. Which networking type did you use?
   (For example: nova-network, Neutron with OpenVSwitch, ...)

   N/A

Logs & Configs
==============

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/757893

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/757894

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.opendev.org/757893
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=770c139edbd2a2c90d6cb2915948e1718dd1deca
Submitter: Zuul
Branch: master

commit 770c139edbd2a2c90d6cb2915948e1718dd1deca
Author: Lee Yarwood <email address hidden>
Date: Tue Oct 13 10:34:27 2020 +0100

    Add regression test for bug #1899649

    Related-Bug: #1899649
    Change-Id: If795b603d812917c9257ee5166a979c1fd6687f3

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 22.1.0

This issue was fixed in the openstack/nova 22.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 23.0.0.0rc1

This issue was fixed in the openstack/nova 23.0.0.0rc1 release candidate.

Lee Yarwood (lyarwood)
Changed in nova:
status: In Progress → Fix Released
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/train)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/768180
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/768194
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/ussuri)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/764625
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/764626
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.