consistency snapshot creation passed even if the volume is in error state

Bug #1592451 reported by Scott DAngelo
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Medium
Nitin Madhok

Bug Description

This is on stable/mitaka with 3par backend.
The expectation is that if the volume is in error, the cgsnapshot creation should fail.

To reproduce:
1) created a cinder type and attached the backend

cinder type-key TestCG set volume_backend_name=3par_iSCSI

2)Created an consistancy group for that volume type

stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder consisgroup-create --name Test_CG TestCG
+-------------------+-------------------------------------------+
| Property | Value |
+-------------------+-------------------------------------------+
| availability_zone | nova |
| created_at | 2016-06-14T10:13:01.000000 |
| description | None |
| id | 8593a4bd-a521-496b-b598-d28a8245b7b7 |
| name | Test_CG |
| status | available |
| volume_types | [u'c4b5ffa4-3dd9-4148-90bb-82f7407221c7'] |
+-------------------+-------------------------------------------+

3) created an volume using the consistancy group and that volume type

stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder create --consisgroup-id 8593a4bd-a521-496b-b598-d28a8245b7b7 --volume-type TestCG --name iscsi_test_cg 1
+--------------------------------+--------------------------------------+
| Property | Value |
+--------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | 8593a4bd-a521-496b-b598-d28a8245b7b7 |
| created_at | 2016-06-14T10:14:22.000000 |
| description | None |
| encrypted | False |
| id | ebeae7ea-3cc1-452b-8ccc-08add818d172 |
| metadata | {} |
| migration_status | None |
| multiattach | False |
| name | iscsi_test_cg |
| os-vol-host-attr:host | ha-volume-manager@iscsi#FC_r6 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 5a97dc3313994311a53128c3a903fc9d |
| replication_status | disabled |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| updated_at | 2016-06-14T10:14:22.000000 |
| user_id | 7851f223718740568b86300760d87bf5 |
| volume_type | TestCG |
+--------------------------------+--------------------------------------+

4) stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder list
+--------------------------------------+-----------+---------------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+---------------+------+-------------+----------+--------------------------------------+

| ebeae7ea-3cc1-452b-8ccc-08add818d172 | available | iscsi_test_cg | 1 | TestCG | false | |
+--------------------------------------+-----------+---------------+------+-------------+----------+--------------------------------------+

5) I reset the state of volume to error state

cinder reset-state --state error ebeae7ea-3cc1-452b-8ccc-08add818d172

6) stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder list
+--------------------------------------+-----------+---------------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+---------------+------+-------------+----------+--------------------------------------+

| ebeae7ea-3cc1-452b-8ccc-08add818d172 | error | iscsi_test_cg | 1 | TestCG | false | |
+--------------------------------------+-----------+---------------+------+-------------+----------+--------------------------------------+

7) Tried to create an usual way snapshot and it failed as expected

stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder snapshot-create ebeae7ea-3cc1-452b-8ccc-08add818d172
ERROR: Invalid volume: Volume ebeae7ea-3cc1-452b-8ccc-08add818d172 status must be available, but current status is: error. (HTTP 400) (Request-ID: req-f54c5d20-8e62-4915-a79f-a4bfed20f208)

8) Now tried to create consistancy snapshot and it passed

stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder cgsnapshot-create --name CG_snap 8593a4bd-a521-496b-b598-d28a8245b7b7
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| consistencygroup_id | 8593a4bd-a521-496b-b598-d28a8245b7b7 |
| created_at | 2016-06-14T10:15:58.000000 |
| description | None |
| id | ef792156-dded-4194-ad99-c25b3b956dc7 |
| name | CG_snap |
| status | creating |
+---------------------+--------------------------------------+

9) The snapshot-list show the snapshot in available status

stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder snapshot-list
+--------------------------------------+--------------------------------------+-----------+----------------+------+
| ID | Volume ID | Status | Name | Size |
+--------------------------------------+--------------------------------------+-----------+----------------+------+

| 41131a45-78e7-49fa-a88a-0fd6e2cccb51 | ebeae7ea-3cc1-452b-8ccc-08add818d172 | available | CG_snap | 1 |

+--------------------------------------+--------------------------------------+-----------+----------------+------+

10) stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder cgsnapshot-list
+--------------------------------------+-----------+---------+
| ID | Status | Name |
+--------------------------------------+-----------+---------+
| ef792156-dded-4194-ad99-c25b3b956dc7 | available | CG_snap |
+--------------------------------------+-----------+---------+

11) stack@hlm:~/scratch/ansible/next/hos/ansible$ cinder cgsnapshot-show ef792156-dded-4194-ad99-c25b3b956dc7
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| consistencygroup_id | 8593a4bd-a521-496b-b598-d28a8245b7b7 |
| created_at | 2016-06-14T10:15:58.000000 |
| description | None |
| id | ef792156-dded-4194-ad99-c25b3b956dc7 |
| name | CG_snap |
| status | available |
+---------------------+--------------------------------------+

Xing Yang (xing-yang)
tags: added: cg
Revision history for this message
Xing Yang (xing-yang) wrote :

Reproduced this in the trunk.

Changed in cinder:
status: New → Confirmed
assignee: nobody → Xing Yang (xing-yang)
importance: Undecided → Medium
Revision history for this message
Gorka Eguileor (gorka) wrote :

Ok, the problem is that we are always passing True for parameter force to `create_snapshots_in_db` in method `create_cgsnapshot` in cinder/consistencygroup/api.py

Revision history for this message
Xing Yang (xing-yang) wrote :

The "force" flag needs to be True because we want to be able to create cgsnapshot when the volumes in the CG are attached (in-use) (We should add a note there for this). Note: The "force" flag is pre-existing for individual snapshot creation, not added for CG. We need to add a condition check in create_snapshots_in_db. If volume status is 'error', we should raise InvalidVolume whether "force" is True or False.

Revision history for this message
Gorka Eguileor (gorka) wrote :

Since `create_snapshots_in_db` is only called from the `create_cgsnapshot` method we should remove the force parameter (it's always True) and we can either add the additional check in `create_snapshots_in_db` after the call to `_create_snapshot_in_db_validate` or modify this validation method to optionally allow failure on error statuses even when it's forced (this should not be the default since this would change normal snapshot behavior).

Nitin Madhok (nmadhok)
Changed in cinder:
assignee: Xing Yang (xing-yang) → Nitin Madhok (nmadhok)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/340563

Changed in cinder:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/340563
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=fb3a51a2b411f05f16a563eadd03eadcf6f17a80
Submitter: Jenkins
Branch: master

commit fb3a51a2b411f05f16a563eadd03eadcf6f17a80
Author: Nitin Madhok <nmadhok@g.clemson.edu>
Date: Mon Jul 11 16:26:41 2016 -0400

    Fixes consistency snapshot creation

    This fixes the bug where creation of consistency snapshot would pass
    even if the volume is in error state. Also adding unit test to
    ensure that the creation of consistency snapshot fails when
    volume is in error status.

    Closes-Bug: #1592451

    Change-Id: I482a7e01f32d72196568348225eb3c77a432a539
    Signed-off-by: Nitin Madhok <nmadhok@g.clemson.edu>

Changed in cinder:
status: In Progress → Fix Released
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/cinder 9.0.0.0b3

This issue was fixed in the openstack/cinder 9.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/366786

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (stable/mitaka)

Reviewed: https://review.openstack.org/366786
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=de954d04c2accb6b2f8aa9e9a59f5c64fe0109bc
Submitter: Jenkins
Branch: stable/mitaka

commit de954d04c2accb6b2f8aa9e9a59f5c64fe0109bc
Author: Nitin Madhok <nmadhok@g.clemson.edu>
Date: Mon Jul 11 16:26:41 2016 -0400

    Fixes consistency snapshot creation

    This fixes the bug where creation of consistency snapshot would pass
    even if the volume is in error state. Also adding unit test to
    ensure that the creation of consistency snapshot fails when
    volume is in error status.

    Made minor changes to unit test for cherry-pick to mitaka.

    Closes-Bug: #1592451

    Change-Id: I482a7e01f32d72196568348225eb3c77a432a539
    Signed-off-by: Nitin Madhok <nmadhok@g.clemson.edu>
    (cherry-picked from commit fb3a51a2b411f05f16a563eadd03eadcf6f17a80)

tags: added: in-stable-mitaka
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/cinder 8.1.1

This issue was fixed in the openstack/cinder 8.1.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.