A race condtion in LVMVolumeDriver create_cloned_volume method

[Rongze Zhu]

When we create a clone volume, LVMVolumeDriver will create a temp snapshot of source volume. The temp snapshot name is "clone-snap-%s" % source_volume['id']. When we create multiple clone volumes of the volume simultaneously, which would trigger the race issuse.

Reproduce:

zhu@dev:~/git/devstack$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| 2130cb05-fc8b-4aef-b883-93e16aeb20c5 | available | None | 1 | None | False | |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
zhu@dev:~/git/devstack$
zhu@dev:~/git/devstack$ cinder create --source-volid=2130cb05-fc8b-4aef-b883-93e16aeb20c5 1
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | False |
| created_at | 2013-07-17T09:52:10.111332 |
| display_description | None |
| display_name | None |
| id | 12bd9108-9adb-41f5-b21e-5993d0936e78 |
| metadata | {} |
| size | 1 |
| snapshot_id | None |
| source_volid | 2130cb05-fc8b-4aef-b883-93e16aeb20c5 |
| status | creating |
| volume_type | None |
+---------------------+--------------------------------------+
zhu@dev:~/git/devstack$ cinder create --source-volid=2130cb05-fc8b-4aef-b883-93e16aeb20c5 1
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | False |
| created_at | 2013-07-17T09:52:13.110964 |
| display_description | None |
| display_name | None |
| id | 881a79f3-c9ba-432e-8546-02923544960a |
| metadata | {} |
| size | 1 |
| snapshot_id | None |
| source_volid | 2130cb05-fc8b-4aef-b883-93e16aeb20c5 |
| status | creating |
| volume_type | None |
+---------------------+--------------------------------------+
zhu@dev:~/git/devstack$
zhu@dev:~/git/devstack$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| 12bd9108-9adb-41f5-b21e-5993d0936e78 | creating | None | 1 | None | False | |
| 2130cb05-fc8b-4aef-b883-93e16aeb20c5 | available | None | 1 | None | False | |
| 881a79f3-c9ba-432e-8546-02923544960a | error | None | 1 | None | False | |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
zhu@dev:~/git/devstack$

The cinder-volume error log:

2013-07-17 17:52:13.497 DEBUG cinder.openstack.common.processutils [req-04f77c08-bccf-4431-b695-e376f9a3ee56 f837a952f4f94967b9cb8a1cfc262589 2a841b64e1e341bd95ebb65bddaa51dd] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvcreate -L 1G --name clone-snap-2130cb05-fc8b-4aef-b883-93e16aeb20c5 --snapshot stack-volumes/volume-2130cb05-fc8b-4aef-b883-93e16aeb20c5 from (pid=8424) execute /opt/stack/cinder/cinder/openstack/common/processutils.py:142
2013-07-17 17:52:13.852 DEBUG cinder.openstack.common.processutils [req-04f77c08-bccf-4431-b695-e376f9a3ee56 f837a952f4f94967b9cb8a1cfc262589 2a841b64e1e341bd95ebb65bddaa51dd] Result was 5 from (pid=8424) execute /opt/stack/cinder/cinder/openstack/common/processutils.py:167
2013-07-17 17:52:13.855 ERROR cinder.volume.driver [req-04f77c08-bccf-4431-b695-e376f9a3ee56 f837a952f4f94967b9cb8a1cfc262589 2a841b64e1e341bd95ebb65bddaa51dd] Recovering from a failed execute. Try number 1
2013-07-17 17:52:13.855 TRACE cinder.volume.driver Traceback (most recent call last):
2013-07-17 17:52:13.855 TRACE cinder.volume.driver File "/opt/stack/cinder/cinder/volume/driver.py", line 103, in _try_execute
2013-07-17 17:52:13.855 TRACE cinder.volume.driver self._execute(*command, **kwargs)
2013-07-17 17:52:13.855 TRACE cinder.volume.driver File "/opt/stack/cinder/cinder/utils.py", line 107, in execute
2013-07-17 17:52:13.855 TRACE cinder.volume.driver description=ex.description)
2013-07-17 17:52:13.855 TRACE cinder.volume.driver ProcessExecutionError: Unexpected error while running command.
2013-07-17 17:52:13.855 TRACE cinder.volume.driver Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvcreate -L 1G --name clone-snap-2130cb05-fc8b-4aef-b883-93e16aeb20c5 --snapshot stack-volumes/volume-2130cb05-fc8b-4aef-b883-93e16aeb20c5
2013-07-17 17:52:13.855 TRACE cinder.volume.driver Exit code: 5
2013-07-17 17:52:13.855 TRACE cinder.volume.driver Stdout: ''
2013-07-17 17:52:13.855 TRACE cinder.volume.driver Stderr: ' Logical volume "clone-snap-2130cb05-fc8b-4aef-b883-93e16aeb20c5" already exists in volume group "stack-volumes"\n'
2013-07-17 17:52:13.855 TRACE cinder.volume.driver

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/37444

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/37444
Committed: http://github.com/openstack/cinder/commit/201890c17e0f29d1ec8e5040206ecd1be09e22f7
Submitter: Jenkins
Branch: master

commit 201890c17e0f29d1ec8e5040206ecd1be09e22f7
Author: Rongze Zhu <email address hidden>
Date: Wed Jul 17 18:15:04 2013 +0800

    Fixes race condition in LVMVolumeDriver create_cloned_volume method

    When we create a clone volume, LVMVolumeDriver will firstly create a temp
    snapshot for copying, the temp snapshot name is
    "clone-snap-%s" % source_volume['id']. When we create multiple clone volumes
    of the same volume simultaneously, which would trigger the race issuse.

    I changes the temp snapshot name template to "clone-snap-%s" % volume['id'],
    so temp snapshot name will be unique.

    Fixes bug #1202139

    Change-Id: I3a4f2585ea9e6f70b6f6f9889e900f19559b8176