volume/snapshots takes very long to delete

[Gaurav Gupta]

I have a large number of snapshots created from a volume. These are COW snapshots so the actual disk space they take is not large, but when I try to delete them using the OSAPI, it takes a very long time to delete each one of them. The volumes and snapshots are based on LVM.

Looking at the code, nova-volumes clears the volume/snapshots using "dd if=/dev/zero of=/dev/mapper/<vol-name>", which is what takes a long time for large volumes/snapshots. This should be done in a lazy fashion (as mentioned in a TODO comment in the code).

[Brian Waldon (bcwaldon)]

Yep, I've seen this, too. It would be great to speed this up, but it isn't clear how to do so.

[Vish Ishaya (vishvananda)]

Perhaps a periodic callback, with a utils.synchronized so only one delete can be happening at once? We could put them into a new state that isn't shown to the api, but not actually mark them deleted until the dd is done.

[Beraldo (beraldo)]

Is there any problem with doing just a "lvremove"?

[Vish Ishaya (vishvananda)]

Beraldo: yes, the lvremove happens after the dd. If you don't dd the data, a subsequent lvcreate can leak the data that was on the old volume. This is a security issue.

[Beraldo (beraldo)]

Understand.

If we can pass this option to user decide ? a -f option for a fast and insecure ? (only lvremove) and a normal option that does a full dd + lvremove ?

[Tushar Patil (tpatil)]

Because of this problem, we cannot contribute volume snapshot blackbox tests to tempest.
In my environment, I have observed deletion of volume is faster than snapshot.

[Pádraig Brady (p-draigbrady)]

https://review.openstack.org/#/c/72656/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/72656
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=71946855591a41dcc87ef59656a8a340774eeaf2
Submitter: Jenkins
Branch: master

commit 71946855591a41dcc87ef59656a8a340774eeaf2
Author: Pádraig Brady <email address hidden>
Date: Tue Feb 11 11:51:39 2014 +0000

    libvirt: support configurable wipe methods for LVM backed instances

    Provide configurable methods to clear these volumes.
    The new 'volume_clear' and 'volume_clear_size' options
    are the same as currently supported by cinder.

    * nova/virt/libvirt/imagebackend.py: Define the new options.
    * nova/virt/libvirt/utils.py (clear_logical_volume): Support the
    new options. Refactor the existing dd method out to
    _zero_logic_volume().
    * nova/tests/virt/libvirt/test_libvirt_utils.py: Add missing test cases
    for the existing clear_logical_volume code, and for the new code
    supporting the new clearing methods.
    * etc/nova/nova.conf.sample: Add the 2 new config descriptions
    to the [libvirt] section.

    Change-Id: I5551197f9ec89ae2f9b051696bccdeb1af2c031f
    Closes-Bug: #889299