NFS doesn't remove snapshots in error status

[Sofia Enriquez]

Description of problem:
While testing NFS snapshot support, on one of the cases snapshot support was set to false. As such a snapshot was created in error state due to lack of backend snapshot support - expected.

When I tried to delete that snapshot it failed (even when using the --force option). Because of that, I can't remove source volume due to the snapshot.

How reproducible:
One quick way to reproduce is to set 'nfs_snapshot_support' to False inside the cinder.conf.

Steps to Reproduce:
Inside the cinder.conf: set 'nfs_snapshot_support' to False.
1. cinder create 1 --name v1
2. cinder snapshot-create v1 --name snap1
The snap should be in 'error' status.
3. cinder snapshot-delete snap1

Actual results:
Snapshot status is 'error_deleting'.

Expected results:
Snapshot delete successfully.

Additional info:
I'm facing this problem in Stein too.

[Sofia Enriquez (lsofia-enriquez)]

https://review.opendev.org/#/c/679138/

[Walt Boring (walter-boring)]

It seems like the real issue here is the create when disabled issue.
Looking at the _check_snapshot_support(), I'm not sure how that doesn't raise
when snaps are disabled.
https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/nfs.py#L549-L555

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/679138
Committed: https://opendev.org/openstack/cinder/commit/5714bdb8447a38440585116f784a322143711959
Submitter: "Zuul (22348)"
Branch: master

commit 5714bdb8447a38440585116f784a322143711959
Author: Sofia Enriquez <email address hidden>
Date: Wed Aug 28 17:38:50 2019 -0300

    Allow removing NFS snapshots in error status

    The NFS driver doesn't allow to delete snapshots in error state when
    snapshot support is disable.

    I'm facing some scenarios where snapshots are enabled then a snapshot
    is created and after that then snapshots are disabled. This results
    in error state snapshots that can't be deleted. Another scenario is
    - with nfs snapshot support disable from the beginning- the API layer
    allows the snapshot to be created but the NFS driver sets the status
    to "error" leaving the snapshot in the DB with error state.
    Because of that, I have snapshots in error state that I'm not able
    to delete.

    The purpose of this fix allow deleting snapshots in error state.
    It makes sense to block creating snapshots when snapshot support is
    False. However, when deleting snapshot we don't want to block
    attempts to delete a broken snapshot DB entry, which is necessary to
    clean up a failed snapshot create.

    Closes-Bug: #1842088
    Change-Id: Ieb19d5e3f58ae2343b6b145772fec33cb7517ab5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/cinder/+/805283

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/805283
Committed: https://opendev.org/openstack/cinder/commit/5a4f3ac53a554ecaa18cbd0a49c0606dff4e1137
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 5a4f3ac53a554ecaa18cbd0a49c0606dff4e1137
Author: Sofia Enriquez <email address hidden>
Date: Wed Aug 28 17:38:50 2019 -0300

    Allow removing NFS snapshots in error status

    The NFS driver doesn't allow to delete snapshots in error state when
    snapshot support is disable.

    I'm facing some scenarios where snapshots are enabled then a snapshot
    is created and after that then snapshots are disabled. This results
    in error state snapshots that can't be deleted. Another scenario is
    - with nfs snapshot support disable from the beginning- the API layer
    allows the snapshot to be created but the NFS driver sets the status
    to "error" leaving the snapshot in the DB with error state.
    Because of that, I have snapshots in error state that I'm not able
    to delete.

    The purpose of this fix allow deleting snapshots in error state.
    It makes sense to block creating snapshots when snapshot support is
    False. However, when deleting snapshot we don't want to block
    attempts to delete a broken snapshot DB entry, which is necessary to
    clean up a failed snapshot create.

    Closes-Bug: #1842088
    Change-Id: Ieb19d5e3f58ae2343b6b145772fec33cb7517ab5
    (cherry picked from commit 5714bdb8447a38440585116f784a322143711959)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/cinder/+/806696

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 18.1.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/806696
Committed: https://opendev.org/openstack/cinder/commit/fb0efc42149fab524a9d3a95f01b7b71e954f314
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit fb0efc42149fab524a9d3a95f01b7b71e954f314
Author: Sofia Enriquez <email address hidden>
Date: Wed Aug 28 17:38:50 2019 -0300

    Allow removing NFS snapshots in error status

    The NFS driver doesn't allow to delete snapshots in error state when
    snapshot support is disable.

    I'm facing some scenarios where snapshots are enabled then a snapshot
    is created and after that then snapshots are disabled. This results
    in error state snapshots that can't be deleted. Another scenario is
    - with nfs snapshot support disable from the beginning- the API layer
    allows the snapshot to be created but the NFS driver sets the status
    to "error" leaving the snapshot in the DB with error state.
    Because of that, I have snapshots in error state that I'm not able
    to delete.

    The purpose of this fix allow deleting snapshots in error state.
    It makes sense to block creating snapshots when snapshot support is
    False. However, when deleting snapshot we don't want to block
    attempts to delete a broken snapshot DB entry, which is necessary to
    clean up a failed snapshot create.

    Closes-Bug: #1842088
    Change-Id: Ieb19d5e3f58ae2343b6b145772fec33cb7517ab5
    (cherry picked from commit 5714bdb8447a38440585116f784a322143711959)
    (cherry picked from commit 5a4f3ac53a554ecaa18cbd0a49c0606dff4e1137)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 19.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/cinder/+/809669

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/809669
Committed: https://opendev.org/openstack/cinder/commit/e9ef536958d9920b70cbbf27bc7faf548ad42d36
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit e9ef536958d9920b70cbbf27bc7faf548ad42d36
Author: Sofia Enriquez <email address hidden>
Date: Wed Aug 28 17:38:50 2019 -0300

    Allow removing NFS snapshots in error status

    The NFS driver doesn't allow to delete snapshots in error state when
    snapshot support is disable.

    I'm facing some scenarios where snapshots are enabled then a snapshot
    is created and after that then snapshots are disabled. This results
    in error state snapshots that can't be deleted. Another scenario is
    - with nfs snapshot support disable from the beginning- the API layer
    allows the snapshot to be created but the NFS driver sets the status
    to "error" leaving the snapshot in the DB with error state.
    Because of that, I have snapshots in error state that I'm not able
    to delete.

    The purpose of this fix allow deleting snapshots in error state.
    It makes sense to block creating snapshots when snapshot support is
    False. However, when deleting snapshot we don't want to block
    attempts to delete a broken snapshot DB entry, which is necessary to
    clean up a failed snapshot create.

    Closes-Bug: #1842088
    Change-Id: Ieb19d5e3f58ae2343b6b145772fec33cb7517ab5
    (cherry picked from commit 5714bdb8447a38440585116f784a322143711959)
    (cherry picked from commit 5a4f3ac53a554ecaa18cbd0a49c0606dff4e1137)
    (cherry picked from commit fb0efc42149fab524a9d3a95f01b7b71e954f314)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/cinder/+/810406

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 16.4.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 17.2.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/810406
Committed: https://opendev.org/openstack/cinder/commit/51a06f3fcc30d0ceb9ccb98560993146e2273090
Submitter: "Zuul (22348)"
Branch: stable/train

commit 51a06f3fcc30d0ceb9ccb98560993146e2273090
Author: Sofia Enriquez <email address hidden>
Date: Wed Aug 28 17:38:50 2019 -0300

    Allow removing NFS snapshots in error status

    The NFS driver doesn't allow to delete snapshots in error state when
    snapshot support is disable.

    I'm facing some scenarios where snapshots are enabled then a snapshot
    is created and after that then snapshots are disabled. This results
    in error state snapshots that can't be deleted. Another scenario is
    - with nfs snapshot support disable from the beginning- the API layer
    allows the snapshot to be created but the NFS driver sets the status
    to "error" leaving the snapshot in the DB with error state.
    Because of that, I have snapshots in error state that I'm not able
    to delete.

    The purpose of this fix allow deleting snapshots in error state.
    It makes sense to block creating snapshots when snapshot support is
    False. However, when deleting snapshot we don't want to block
    attempts to delete a broken snapshot DB entry, which is necessary to
    clean up a failed snapshot create.

    Closes-Bug: #1842088
    Change-Id: Ieb19d5e3f58ae2343b6b145772fec33cb7517ab5
    (cherry picked from commit 5714bdb8447a38440585116f784a322143711959)
    (cherry picked from commit 5a4f3ac53a554ecaa18cbd0a49c0606dff4e1137)
    (cherry picked from commit fb0efc42149fab524a9d3a95f01b7b71e954f314)
    (cherry picked from commit e9ef536958d9920b70cbbf27bc7faf548ad42d36)