Scope of image volume cache lock is too broad

[Alan Bishop]

A synchronization lock was added to fix a problem that allowed multiple redundant entries in the image volume cache [1],[2].

[1] https://bugs.launchpad.net/cinder/+bug/1649636
[2] https://review.openstack.org/446590

Unfortunately, the scope of the lock is too broad and impacts performance in a couple of scenarios.

1) When the cache is not enabled, simultaneous attempts to download the image from glance are serialized.
2) Even when the volume image is cached, simultaneous requests to create a volume from the image are serialized.

The scope of the lock needs to be reduced to only serialize the portion of the process that results in an entry being added to the cache.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/556607

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/556607
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=a351cd01c0bf937e82efc3e1256da479be24cfcf
Submitter: Zuul
Branch: master

commit a351cd01c0bf937e82efc3e1256da479be24cfcf
Author: Alan Bishop <email address hidden>
Date: Fri Mar 23 19:36:16 2018 +0000

    Reduce scope of the lock for image volume cache

    Refactor the code that creates a volume from a downloaded glance image
    to minimize the scope of the lock that prevents multiple entries in the
    volume image cache. Now the lock serializes only the portion of the
    code that causes the cache entry to be created. Locking is minimized
    when the volume is already cached, or when the volume won't be cached.

    Closes-Bug: #1758414
    Change-Id: I547fb4bcdd4783225b8ca96d157c61ca3bcf4ef4

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/557318

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/557318
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=bc05efc79aaedd06308f7f173df09fd6a23aa2b3
Submitter: Zuul
Branch: stable/queens

commit bc05efc79aaedd06308f7f173df09fd6a23aa2b3
Author: Alan Bishop <email address hidden>
Date: Fri Mar 23 19:36:16 2018 +0000

    Reduce scope of the lock for image volume cache

    Refactor the code that creates a volume from a downloaded glance image
    to minimize the scope of the lock that prevents multiple entries in the
    volume image cache. Now the lock serializes only the portion of the
    code that causes the cache entry to be created. Locking is minimized
    when the volume is already cached, or when the volume won't be cached.

    Closes-Bug: #1758414
    Change-Id: I547fb4bcdd4783225b8ca96d157c61ca3bcf4ef4
    (cherry picked from commit a351cd01c0bf937e82efc3e1256da479be24cfcf)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/558556

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/558556
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=a98afbc53a6074e07eb476c8f113418be8a189a4
Submitter: Zuul
Branch: stable/pike

commit a98afbc53a6074e07eb476c8f113418be8a189a4
Author: Alan Bishop <email address hidden>
Date: Fri Mar 23 19:36:16 2018 +0000

    Reduce scope of the lock for image volume cache

    Refactor the code that creates a volume from a downloaded glance image
    to minimize the scope of the lock that prevents multiple entries in the
    volume image cache. Now the lock serializes only the portion of the
    code that causes the cache entry to be created. Locking is minimized
    when the volume is already cached, or when the volume won't be cached.

    Closes-Bug: #1758414
    Change-Id: I547fb4bcdd4783225b8ca96d157c61ca3bcf4ef4
    (cherry picked from commit a351cd01c0bf937e82efc3e1256da479be24cfcf)
    (cherry picked from commit bc05efc79aaedd06308f7f173df09fd6a23aa2b3)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/560891

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 13.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 12.0.1 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/560891
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=11697ddde34c613dd91203f45090750d481a18d4
Submitter: Zuul
Branch: stable/ocata

commit 11697ddde34c613dd91203f45090750d481a18d4
Author: Alan Bishop <email address hidden>
Date: Fri Mar 23 19:36:16 2018 +0000

    Reduce scope of the lock for image volume cache

    Refactor the code that creates a volume from a downloaded glance image
    to minimize the scope of the lock that prevents multiple entries in the
    volume image cache. Now the lock serializes only the portion of the
    code that causes the cache entry to be created. Locking is minimized
    when the volume is already cached, or when the volume won't be cached.

    Closes-Bug: #1758414
    Change-Id: I547fb4bcdd4783225b8ca96d157c61ca3bcf4ef4
    (cherry picked from commit a351cd01c0bf937e82efc3e1256da479be24cfcf)
    (cherry picked from commit bc05efc79aaedd06308f7f173df09fd6a23aa2b3)
    (cherry picked from commit a98afbc53a6074e07eb476c8f113418be8a189a4)
    Conflicts:
     cinder/tests/unit/volume/flows/test_create_volume_flow.py
     cinder/volume/flows/manager/create_volume.py

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 10.0.7 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 11.1.1 release.