status=None while delete is in progress

Bug #1503390 reported by Mark Sturdevant
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
High
NidhiMittalHada

Bug Description

When deleting snapshots and getting snapshot lists, testing has shown that the status of a snapshot may be None. It looks like this happens when the snapshot instance is has been deleted, but the snapshot itself has not yet been deleted. This would happen in a very small window.

A snapshot with no instances should not be included in the list -- or perhaps it could be in the list, but should have a status of "deleting".

It seems the status handling and/or the order of deletion of the snapshot and its final (only) instance need to be fixed.

The same problem has been caught with deleted shares returning a status of None.

Changed in manila:
importance: Undecided → High
status: New → Triaged
milestone: none → mitaka-1
Revision history for this message
Mark Sturdevant (mark-sturdevant) wrote :

This has been seen with shares as well as snapshots. Our "instance" logic needs to be improved.

summary: - snapshot status=None while delete is in progress
+ status=None while delete is in progress
description: updated
Changed in manila:
assignee: nobody → NidhiMittalHada (nidhimittal19)
Revision history for this message
NidhiMittalHada (nidhimittal19) wrote :

working on it

Revision history for this message
NidhiMittalHada (nidhimittal19) wrote :

reproduced the problem.

as its a small window

manila/db/sqlalchemy/api.py
@require_context
def share_snapshot_destroy(context, snapshot_id):
    session = get_session()
    with session.begin():
        snapshot_ref = share_snapshot_get(context, snapshot_id,
                                          session=session)
        snapshot_ref.instance.soft_delete(session=session, update_status=True)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>HERE !!!
        snapshot_ref.soft_delete(session=session)

difficult to reproduce.

used database direct query to visualize the problem.

stack@controller:~/devstack$ manila snapshot-show f5677994-9d4b-4261-b1dc-a13a3c78eb18
+-------------+--------------------------------------+
| Property | Value |
+-------------+--------------------------------------+
| status | available |
| share_id | 44de46a2-72e0-475d-a997-8afa8c93c0d1 |
| name | None |
| created_at | 2015-12-21T09:35:33.000000 |
| share_proto | NFS |
| id | f5677994-9d4b-4261-b1dc-a13a3c78eb18 |
| size | 1 |
| share_size | 1 |
| description | None |
+-------------+--------------------------------------+

>>>>>>>>>>>>>>>>.after injecting the problem ................

stack@controller:~/devstack$ manila snapshot-show f5677994-9d4b-4261-b1dc-a13a3c78eb18
+-------------+--------------------------------------+
| Property | Value |
+-------------+--------------------------------------+
| status | None |
| share_id | 44de46a2-72e0-475d-a997-8afa8c93c0d1 |
| name | None |
| created_at | 2015-12-21T09:35:33.000000 |
| share_proto | NFS |
| id | f5677994-9d4b-4261-b1dc-a13a3c78eb18 |
| size | 1 |
| share_size | 1 |
| description | None |
+-------------+--------------------------------------+
stack@controller:~/devstack$ manila snapshot-list
+--------------------------------------+--------------------------------------+--------+------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+--------+------+------------+
| f5677994-9d4b-4261-b1dc-a13a3c78eb18 | 44de46a2-72e0-475d-a997-8afa8c93c0d1 | None | None | 1 |
+--------------------------------------+--------------------------------------+--------+------+------------+

working on it ...

Changed in manila:
milestone: mitaka-1 → mitaka-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/261289

Changed in manila:
status: Triaged → In Progress
Changed in manila:
milestone: mitaka-2 → mitaka-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.openstack.org/261289
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=12548c9cea3c81ddd68e75c9433e8d6c735ce9b9
Submitter: Jenkins
Branch: master

commit 12548c9cea3c81ddd68e75c9433e8d6c735ce9b9
Author: nidhimittalhada <email address hidden>
Date: Thu Dec 24 17:31:28 2015 +0530

    Hide snapshots with no instances from listing

    When a snapshot is being destroyed; for a small window of time
    between the deletion of all of its instances and the snapshot ref
    itself, the 'status' of the snapshot becomes 'None'. This provides
    a bad user experience if a list/show query is made within that
    window.

    Fix this problem by filtering out snapshots that have this 'invalid'
    status.

    Corrected a few unit tests, to use 'status' as a manadatory
    field in snapshot record.

    Change-Id: I2ec83eecc9788278459846502625df7fde1fccbd
    Closes-Bug: #1503390

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/manila 2.0.0.0b3

This issue was fixed in the openstack/manila 2.0.0.0b3 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.