GlusterFS: operations that modify data require locking

Bug #1251425 reported by Eric Harney
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
High
Eric Harney
Havana
Fix Released
High
Eric Harney

Bug Description

It is currently possible for multiple threads to run create_snapshot operations at a time. This may result in a corrupt/incomplete snapshot chain as qcow2 backing file pointers and snapshot info file do not stay in sync with each other.

To demonstrate this, add a time.sleep(15) call inside of _create_snapshot() immediately after the call self._create_qcow2_snap_file() and run many snapshot-create operations at once, then attempt to delete all of the snapshots that have been created.

Eric Harney (eharney)
tags: added: drivers glusterfs
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/56462

Changed in cinder:
assignee: nobody → Eric Harney (eharney)
status: New → In Progress
Eric Harney (eharney)
tags: added: havana-backport-potential
Eric Harney (eharney)
Changed in cinder:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/56462
Committed: http://github.com/openstack/cinder/commit/8a1fd8889170c289910944db016a4206fe5e1e32
Submitter: Jenkins
Branch: master

commit 8a1fd8889170c289910944db016a4206fe5e1e32
Author: Eric Harney <email address hidden>
Date: Thu Nov 14 15:51:15 2013 -0500

    GlusterFS: Synchronize operations that manipulate qcow2 data

    Operations that modify qcow2 chains or the GlusterFS driver's
    snapshot info file should use locking so that more than one thread
    cannot run these operations at the same time.

    Without this it is possible for multiple threads to concurrently
    modify snapshot information resulting in an incomplete snapshot
    chain.

    Closes-Bug: #1251425

    Change-Id: I93c6ffecd9aec38560cb19085243390b9120b363

Changed in cinder:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/57943

Thierry Carrez (ttx)
Changed in cinder:
milestone: none → icehouse-1
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (stable/havana)

Reviewed: https://review.openstack.org/57943
Committed: http://github.com/openstack/cinder/commit/009b0544a2be24779192b11d159032f27d83e272
Submitter: Jenkins
Branch: stable/havana

commit 009b0544a2be24779192b11d159032f27d83e272
Author: Eric Harney <email address hidden>
Date: Thu Nov 14 15:51:15 2013 -0500

    GlusterFS: Synchronize operations that manipulate qcow2 data

    Operations that modify qcow2 chains or the GlusterFS driver's
    snapshot info file should use locking so that more than one thread
    cannot run these operations at the same time.

    Without this it is possible for multiple threads to concurrently
    modify snapshot information resulting in an incomplete snapshot
    chain.

    Closes-Bug: #1251425

    (cherry picked from commit 8a1fd8889170c289910944db016a4206fe5e1e32)
    Change-Id: I93c6ffecd9aec38560cb19085243390b9120b363

tags: added: in-stable-havana
Alan Pevec (apevec)
tags: removed: havana-backport-potential in-stable-havana
Thierry Carrez (ttx)
Changed in cinder:
milestone: icehouse-1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.