Duplicate BlockDeviceMapping if attaching volumes too fast

[Felix Huettner]

Description
===========
When attaching a volume to a running instance the nova-api validates that
the volume is not already attached to the instance. However nova-compute
is responsible for actually creating the BDM entry in the database.
If sending attach requests fast enough it can be possible that the same
"attach_volume" request can be sent to nova-compute for the same
volume/instance combination.

Steps to reproduce
==================
* Create an Instance and a volume
* run in 2 terminals in parallel "openstack server add volume <instance> <volume>"
* when being fast enough this results in "openstack server show" to report the volume id twice

Expected result
===============
The volume is attached to the instance just once

Actual result
=============
The volume is attached to the instance two (or with more parallelity) or more times.

Environment
===========
1. Happens on master till queens (possibly also earlier releases)

2. Which hypervisor did you use?
   Indepentent of Hypervisor, observed with Libvirt

2. Which storage type did you use?
   Independent of Storage, observed with ceph + nfs

3. Which networking type did you use?
   Independent of Network, observed with ovs

[Felix Huettner (felix.huettner)]

Fix is being developed in https://review.opendev.org/c/openstack/nova/+/801990

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/802011
Committed: https://opendev.org/openstack/nova/commit/2ffd9738602531e93495a1feca76bbb687c3e72c
Submitter: "Zuul (22348)"
Branch: master

commit 2ffd9738602531e93495a1feca76bbb687c3e72c
Author: Lee Yarwood <email address hidden>
Date: Fri Jul 23 11:12:56 2021 +0100

    Add functional test for bug 1937375

    Related-Bug: #1937375
    Change-Id: I332d4f33ea6b9506cc24ac12e5c0994f208a3107

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/801990
Committed: https://opendev.org/openstack/nova/commit/2209b0007fe85d7c5439e0bfdfe2120c63898fa2
Submitter: "Zuul (22348)"
Branch: master

commit 2209b0007fe85d7c5439e0bfdfe2120c63898fa2
Author: Felix Huettner <email address hidden>
Date: Fri Jul 23 10:43:32 2021 +0200

    compute: Avoid duplicate BDMs during reserve_block_device_name

    When attaching a volume to a running instance the nova-api validates
    that the volume is not already attached to the instance. However
    nova-compute is responsible for actually creating the BDM entry in the
    database. If sending attach requests fast enough it can be possible
    that the same "attach_volume" request can be sent to nova-compute for
    the same volume/instance combination.

    To work around this we add a check in nova-compute to validate that
    the volume has not been attached in the mean time.

    Closes-Bug: #1937375
    Change-Id: I92f35514efddcb071c7094370b79d91d34c5bc72

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/803717

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/803718

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/803720

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/803721

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/803722

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/803723

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/803727

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/803728

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/803762

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/803763

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.opendev.org/c/openstack/nova/+/803764

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/c/openstack/nova/+/803765

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.opendev.org/c/openstack/nova/+/803766

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/c/openstack/nova/+/803767

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 24.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/803717
Committed: https://opendev.org/openstack/nova/commit/7a9e3dcd172f187cf93f406b09f49d2ded9bd90d
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 7a9e3dcd172f187cf93f406b09f49d2ded9bd90d
Author: Lee Yarwood <email address hidden>
Date: Fri Jul 23 11:12:56 2021 +0100

    Add functional test for bug 1937375

    Related-Bug: #1937375
    Change-Id: I332d4f33ea6b9506cc24ac12e5c0994f208a3107
    (cherry picked from commit 2ffd9738602531e93495a1feca76bbb687c3e72c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/803718
Committed: https://opendev.org/openstack/nova/commit/2bee83b8a980b2bd9e276b75aa74253f8c0d0a70
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 2bee83b8a980b2bd9e276b75aa74253f8c0d0a70
Author: Felix Huettner <email address hidden>
Date: Fri Jul 23 10:43:32 2021 +0200

    compute: Avoid duplicate BDMs during reserve_block_device_name

    When attaching a volume to a running instance the nova-api validates
    that the volume is not already attached to the instance. However
    nova-compute is responsible for actually creating the BDM entry in the
    database. If sending attach requests fast enough it can be possible
    that the same "attach_volume" request can be sent to nova-compute for
    the same volume/instance combination.

    To work around this we add a check in nova-compute to validate that
    the volume has not been attached in the mean time.

    Closes-Bug: #1937375
    Change-Id: I92f35514efddcb071c7094370b79d91d34c5bc72
    (cherry picked from commit 2209b0007fe85d7c5439e0bfdfe2120c63898fa2)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 23.2.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/queens
Review: https://review.opendev.org/c/openstack/nova/+/803767
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/queens
Review: https://review.opendev.org/c/openstack/nova/+/803766
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/rocky
Review: https://review.opendev.org/c/openstack/nova/+/803765
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/rocky
Review: https://review.opendev.org/c/openstack/nova/+/803764
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/803762
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/803763
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/803728
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/803727
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/803722
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/803723
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/803721
Reason: stable/victoria branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/victoria if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/803720
Reason: stable/victoria branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/victoria if you want to further work on this patch.