When a a storagegroup is destroyed because it's empty, VNX driver doesn't recreate it upon subsequent attachment

Bug #1817385 reported by David Vallee Delisle on 2019-02-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Undecided
Ryan Liang

Bug Description

Description of problem:

When we enable destroy_empty_storage_group and a storage group is destroyed after it's emptied, the VNX driver or storops cache isn't updated and the driver doesn't recreate the storage group upon subsequent attachment.

I'm not sure if it's a requisite but we had iscsi auto deregistration enabled as well.

Version-Release number of selected component (if applicable):
openstack-cinder-9.1.4-41

How reproducible:
100%

Steps to Reproduce:
1. Enable destroy_empty_storage_group initiator_auto_deregistration in vnx driver configuration
2. Delete attachment of all volumes on a compute node until it's deregistered by the driver
3. Attempt to re-attach a volume on deregistered compute node.

Actual results:
~~~
2019-02-25 17:11:30.089 10661 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: Bad or unexpected response from the storage volume backend API: Unable to fetch connection information from backend: Error: storagegroup command failed
2019-02-25 17:11:30.089 10661 ERROR oslo_messaging.rpc.server The group name or UID does not match any storage groups for this array
~~~

Expected results:
It should work

description: updated
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/639241

Changed in cinder:
assignee: nobody → Ryan Liang (ryan-liang)
status: New → In Progress

Reviewed: https://review.openstack.org/639241
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=eefd8302b3a01fd5cff0169b6eca0588975ff244
Submitter: Zuul
Branch: master

commit eefd8302b3a01fd5cff0169b6eca0588975ff244
Author: Ryan Liang <email address hidden>
Date: Mon Feb 25 17:43:46 2019 +0800

    VNX: update sg in cache

    When `destroy_empty_storage_group` is enabled, the sg will be deleted
    after the last lun is detached from it. The issue is that the sg cache
    isn't updated, which caused the failure of subsequent attaching volumes
    to the same host because the sg in the cache will be used directly which
    doesn't exist any more.

    The fix is updating the sg with poll after deleting it. This doesn't
    delete the sg from cache explicitly but makes sure the sg in cache with
    latest status `not-exist`. Subsequent attaching will create the sg if it
    is `not-exist`.

    Closes-bug: #1817385
    Change-Id: Ibb39879a77c97c6a5d885461e93116810d16b265

Changed in cinder:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/640120
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=1f57f502c893a840ee5021fbca36ae95eed50e73
Submitter: Zuul
Branch: stable/rocky

commit 1f57f502c893a840ee5021fbca36ae95eed50e73
Author: Ryan Liang <email address hidden>
Date: Mon Feb 25 17:43:46 2019 +0800

    VNX: update sg in cache

    When `destroy_empty_storage_group` is enabled, the sg will be deleted
    after the last lun is detached from it. The issue is that the sg cache
    isn't updated, which caused the failure of subsequent attaching volumes
    to the same host because the sg in the cache will be used directly which
    doesn't exist any more.

    The fix is updating the sg with poll after deleting it. This doesn't
    delete the sg from cache explicitly but makes sure the sg in cache with
    latest status `not-exist`. Subsequent attaching will create the sg if it
    is `not-exist`.

    Closes-bug: #1817385
    Change-Id: Ibb39879a77c97c6a5d885461e93116810d16b265
    (cherry picked from commit eefd8302b3a01fd5cff0169b6eca0588975ff244)

tags: added: in-stable-rocky

Reviewed: https://review.openstack.org/642613
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=29d42fa73448b3d23d9da4db7b2f407694681ece
Submitter: Zuul
Branch: stable/queens

commit 29d42fa73448b3d23d9da4db7b2f407694681ece
Author: Ryan Liang <email address hidden>
Date: Mon Feb 25 17:43:46 2019 +0800

    VNX: update sg in cache

    When `destroy_empty_storage_group` is enabled, the sg will be deleted
    after the last lun is detached from it. The issue is that the sg cache
    isn't updated, which caused the failure of subsequent attaching volumes
    to the same host because the sg in the cache will be used directly which
    doesn't exist any more.

    The fix is updating the sg with poll after deleting it. This doesn't
    delete the sg from cache explicitly but makes sure the sg in cache with
    latest status `not-exist`. Subsequent attaching will create the sg if it
    is `not-exist`.

    Closes-bug: #1817385
    Change-Id: Ibb39879a77c97c6a5d885461e93116810d16b265
    (cherry picked from commit eefd8302b3a01fd5cff0169b6eca0588975ff244)
    (cherry picked from commit 1f57f502c893a840ee5021fbca36ae95eed50e73)

tags: added: in-stable-queens

This issue was fixed in the openstack/cinder 14.0.0.0rc1 release candidate.

This issue was fixed in the openstack/cinder 13.0.4 release.

This issue was fixed in the openstack/cinder 12.0.6 release.

Reviewed: https://review.openstack.org/643795
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=dcfc2f3d5ac69e0fd0c6ecbdd6ce26ed1cecd96c
Submitter: Zuul
Branch: stable/pike

commit dcfc2f3d5ac69e0fd0c6ecbdd6ce26ed1cecd96c
Author: Ryan Liang <email address hidden>
Date: Mon Feb 25 17:43:46 2019 +0800

    VNX: update sg in cache

    When `destroy_empty_storage_group` is enabled, the sg will be deleted
    after the last lun is detached from it. The issue is that the sg cache
    isn't updated, which caused the failure of subsequent attaching volumes
    to the same host because the sg in the cache will be used directly which
    doesn't exist any more.

    The fix is updating the sg with poll after deleting it. This doesn't
    delete the sg from cache explicitly but makes sure the sg in cache with
    latest status `not-exist`. Subsequent attaching will create the sg if it
    is `not-exist`.

    Closes-bug: #1817385
    Change-Id: Ibb39879a77c97c6a5d885461e93116810d16b265
    (cherry picked from commit eefd8302b3a01fd5cff0169b6eca0588975ff244)
    (cherry picked from commit 1f57f502c893a840ee5021fbca36ae95eed50e73)
    (cherry picked from commit 29d42fa73448b3d23d9da4db7b2f407694681ece)

tags: added: in-stable-pike

Reviewed: https://review.openstack.org/648051
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=187016c2852533c1806c7fdb34aa8dc6dfcd528e
Submitter: Zuul
Branch: stable/ocata

commit 187016c2852533c1806c7fdb34aa8dc6dfcd528e
Author: Ryan Liang <email address hidden>
Date: Mon Feb 25 17:43:46 2019 +0800

    VNX: update sg in cache

    When `destroy_empty_storage_group` is enabled, the sg will be deleted
    after the last lun is detached from it. The issue is that the sg cache
    isn't updated, which caused the failure of subsequent attaching volumes
    to the same host because the sg in the cache will be used directly which
    doesn't exist any more.

    The fix is updating the sg with poll after deleting it. This doesn't
    delete the sg from cache explicitly but makes sure the sg in cache with
    latest status `not-exist`. Subsequent attaching will create the sg if it
    is `not-exist`.

    Closes-bug: #1817385
    Change-Id: Ibb39879a77c97c6a5d885461e93116810d16b265
    (cherry picked from commit eefd8302b3a01fd5cff0169b6eca0588975ff244)
    (cherry picked from commit 1f57f502c893a840ee5021fbca36ae95eed50e73)
    (cherry picked from commit 29d42fa73448b3d23d9da4db7b2f407694681ece)
    (cherry picked from commit dcfc2f3d5ac69e0fd0c6ecbdd6ce26ed1cecd96c)

tags: added: in-stable-ocata

This issue was fixed in the openstack/cinder 11.2.2 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers