Volume/snapshot manage quota commit twice

[James Conroy]

When the snapshot-manage feature is used, the snapshots quota increments by 2 instead of 1. If snapshot-unmanage is used then the quota decrements by 1 (expected). This means that a net number of 1 snapshots are created when a snapshot is managed and afterwards unmanaged. This will force a user to bypass their quota faster, and bypass it even if they currently have zero snapshots managed.

Reproduced on a fresh DevStack (branch master):

jay@JubuntuVM:~/devstack$ sudo lvcreate --size 1G --name jvol stack-volumes-lvmdriver-1
  Logical volume "jvol" created

jay@JubuntuVM:~/devstack$ sudo lvcreate --size 1G --snapshot --name jvolsnap /dev/stack-volumes-lvmdriver-1/jvol
  Logical volume "jvolsnap" created

jay@JubuntuVM:~/devstack$ cinder list
+----+--------+------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+----+--------+------+------+-------------+----------+-------------+
+----+--------+------+------+-------------+----------+-------------+

jay@JubuntuVM:~/devstack$ cinder quota-usage a49201ef2cd748609a782daa6c8fccc2
+-----------------------+--------+----------+-------+
| Type | In_use | Reserved | Limit |
+-----------------------+--------+----------+-------+
| backup_gigabytes | 0 | 0 | 1000 |
| backups | 0 | 0 | 10 |
| gigabytes | 0 | 0 | 1000 |
| gigabytes_lvmdriver-1 | 0 | 0 | -1 |
| per_volume_gigabytes | 0 | 0 | -1 |
| snapshots | 0 | 0 | 10 |
| snapshots_lvmdriver-1 | 0 | 0 | -1 |
| volumes | 0 | 0 | 10 |
| volumes_lvmdriver-1 | 0 | 0 | -1 |
+-----------------------+--------+----------+-------+

jay@JubuntuVM:~/devstack$ cinder manage JubuntuVM@lvmdriver-1#lvmdriver-1 jvol
+--------------------------------+--------------------------------------+
| Property | Value |
+--------------------------------+--------------------------------------+
[...]
+--------------------------------+--------------------------------------+

jay@JubuntuVM:~/devstack$ cinder list
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| 46db1ddb-2ab2-4487-9403-a02e66970b0c | available | - | 1 | - | false | |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+

jay@JubuntuVM:~/devstack$ cinder quota-usage a49201ef2cd748609a782daa6c8fccc2
+-----------------------+--------+----------+-------+
| Type | In_use | Reserved | Limit |
+-----------------------+--------+----------+-------+
| backup_gigabytes | 0 | 0 | 1000 |
| backups | 0 | 0 | 10 |
| gigabytes | 1 | 0 | 1000 |
| gigabytes_lvmdriver-1 | 0 | 0 | -1 |
| per_volume_gigabytes | 0 | 0 | -1 |
| snapshots | 0 | 0 | 10 |
| snapshots_lvmdriver-1 | 0 | 0 | -1 |
| volumes | 2 | 0 | 10 |
| volumes_lvmdriver-1 | 0 | 0 | -1 |
+-----------------------+--------+----------+-------+

jay@JubuntuVM:~/devstack$ cinder snapshot-manage 46db1ddb-2ab2-4487-9403-a02e66970b0c jvolsnap
+--------------------------------------------+--------------------------------------+
| Property | Value |
+--------------------------------------------+--------------------------------------+
| created_at | 2016-06-09T08:23:47.000000 |
| description | None |
| id | 51931ed2-8cfe-49d7-ae71-c650a11f8846 |
| metadata | {} |
| name | None |
| os-extended-snapshot-attributes:progress | 0% |
| os-extended-snapshot-attributes:project_id | a49201ef2cd748609a782daa6c8fccc2 |
| size | 1 |
| status | available |
| updated_at | 2016-06-09T08:23:48.000000 |
| volume_id | 46db1ddb-2ab2-4487-9403-a02e66970b0c |
+--------------------------------------------+--------------------------------------+

jay@JubuntuVM:~/devstack$ cinder snapshot-list
+--------------------------------------+--------------------------------------+-----------+------+------+
| ID | Volume ID | Status | Name | Size |
+--------------------------------------+--------------------------------------+-----------+------+------+
| 51931ed2-8cfe-49d7-ae71-c650a11f8846 | 46db1ddb-2ab2-4487-9403-a02e66970b0c | available | - | 1 |
+--------------------------------------+--------------------------------------+-----------+------+------+

jay@JubuntuVM:~/devstack$ cinder quota-usage a49201ef2cd748609a782daa6c8fccc2
+-----------------------+--------+----------+-------+
| Type | In_use | Reserved | Limit |
+-----------------------+--------+----------+-------+
| backup_gigabytes | 0 | 0 | 1000 |
| backups | 0 | 0 | 10 |
| gigabytes | 2 | 0 | 1000 |
| gigabytes_lvmdriver-1 | 0 | 0 | -1 |
| per_volume_gigabytes | 0 | 0 | -1 |
| snapshots | 2 | 0 | 10 |
| snapshots_lvmdriver-1 | 0 | 0 | -1 |
| volumes | 2 | 0 | 10 |
| volumes_lvmdriver-1 | 0 | 0 | -1 |
+-----------------------+--------+----------+-------+

[Sheel Rana (ranasheel2000)]

Dear Jay,

If I am right, this is already fixed by Cao Shufeng.
Let me check, will share the bug id.

[Sheel Rana (ranasheel2000)]

Yes, here is the fix.
https://review.openstack.org/#/c/298522/

remove extra quota commit of manage snapshot

When manage an existing snapshot of a volume, cinder-api and cinder-vol
both commit quota. This result in an extra commit.This change remove
the extra commit in the cinder-api.

Change-Id: Ic93ceebfcd9703549eb5f6b41e404a36ce24daae
Closes-bug: #1562830

[Sheel Rana (ranasheel2000)]

For now claiming this duplicate of https://bugs.launchpad.net/cinder/+bug/1562830.
Please reopen if I am missing anything you wants to point through this bug.

Note: We should cherry pick in case we want to fix in previous versions than master. We need not to raise bug for that.

[James Conroy (jayconroy)]

This bug is not a duplicate, it can be reproduced with a fresh install of DevStack with 0 volumes and 0 snapshots using a LVM backend volume. The quota bug also affects volumes. Cannot seem to reproduce the bug after reproducing it the first time.

[James Conroy (jayconroy)]

Reproduced on a fresh DevStack (branch master):

jay@JubuntuVM:~/devstack$ sudo lvcreate --size 1G --name jvol stack-volumes-lvmdriver-1
  Logical volume "jvol" created

jay@JubuntuVM:~/devstack$ sudo lvcreate --size 1G --snapshot --name jvolsnap /dev/stack-volumes-lvmdriver-1/jvol
  Logical volume "jvolsnap" created

jay@JubuntuVM:~/devstack$ cinder list
+----+--------+------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+----+--------+------+------+-------------+----------+-------------+
+----+--------+------+------+-------------+----------+-------------+

jay@JubuntuVM:~/devstack$ cinder quota-usage a49201ef2cd748609a782daa6c8fccc2
+-----------------------+--------+----------+-------+
| Type | In_use | Reserved | Limit |
+-----------------------+--------+----------+-------+
| backup_gigabytes | 0 | 0 | 1000 |
| backups | 0 | 0 | 10 |
| gigabytes | 0 | 0 | 1000 |
| gigabytes_lvmdriver-1 | 0 | 0 | -1 |
| per_volume_gigabytes | 0 | 0 | -1 |
| snapshots | 0 | 0 | 10 |
| snapshots_lvmdriver-1 | 0 | 0 | -1 |
| volumes | 0 | 0 | 10 |
| volumes_lvmdriver-1 | 0 | 0 | -1 |
+-----------------------+--------+----------+-------+

jay@JubuntuVM:~/devstack$ cinder manage JubuntuVM@lvmdriver-1#lvmdriver-1 jvol
+--------------------------------+--------------------------------------+
| Property | Value |
+--------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2016-06-09T08:22:53.000000 |
| description | None |
| encrypted | False |
| id | 46db1ddb-2ab2-4487-9403-a02e66970b0c |
| metadata | {} |
| migration_status | None |
| multiattach | False |
| name | None |
| os-vol-host-attr:host | JubuntuVM@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | a49201ef2cd748609a782daa6c8fccc2 |
| replication_status | None |
| size | 0 |
| snapshot_id | None |
| source_volid | None |
| ...

[Sheel Rana (ranasheel2000)]

Jay,

Quite interesting finding..
Please go ahead with fix..

[James Conroy (jayconroy)]

Another thing to note is, if a cinder volume is created first and then the LVM volume managed, the volume quota bug does not occur (total 2 volumes = correct).

[Mitsuhiro Tanino (mitsuhiro-tanino)]

As Jay said, this happens only when,
- No volumes are created on Cinder
- Manage volume, and manage snapshot
and then invalid quota count happens.

[wangxiyuan (wangxiyuan)]

Can't reproduce in Pike. I guess it has been fixed already.

[TommyLike (hu-husheng)]

It's a valid bug. It appears if you try to manage volume or snapshots when there is not any responding quota_usage records in database, it's related to the quota SYNC mechanism.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/521044

[James Conroy (jayconroy)]

Thanks for proposing a fix. If merged, it'd be great to get the fix cherry picked. Thanks

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521044
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=e72f0fdf2678482723e95bcc89a8c2117865c8a7
Submitter: Zuul
Branch: master

commit e72f0fdf2678482723e95bcc89a8c2117865c8a7
Author: TommyLike <email address hidden>
Date: Fri Nov 17 15:41:23 2017 +0800

    Handle quota in 2 phases when managing resource

    Cinder would commit quota twice when managing resource
    in a clean environment that the corresponding
    'quota_usage' record is empty. This is all because of
    we create db entity before reservation, hence the SYNC
    mechanism would refresh new record into quota_usage
    before actually reserved.

    This patch fix this issue by introducing 2 phases
    reserve&commit, the latter only intends to update
    the actual size.

    Closes-Bug: #1587376
    Change-Id: I79940e534ec03f2d327e8a7e14e45bc93ae41b0c

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 12.0.0.0b2 development milestone.