Cinder test is not deleting a volume at tearDown

[Lucio Seki]

VolumeV1CloneTest.test_create_from_bootable_volume creates some volumes, but it doesn't delete one of them on tearDown, leaving a trash volume every time the test is run.

To check if this is happening with your storage, look at scree-c-vol log, filter it by "volume successfully" string (in vim, :v/volume successfully/d), and find a "Created volume successfully" message without corresponding "Deleted volume successfully" message.

To verify which test left this volume, find the closest timestamp of the "Created volume successfully" message at tempest log.

[Lucio Seki (lseki)]

The issue is reproducible by running only this test, exporting the following variable:

export DEVSTACK_GATE_TEMPEST_REGEX='^(?=.*VolumesV1CloneTest\.test_create_from_bootable_volume).*'

[Matthew Treinish (treinish)]

I traced through the test code and the only 2 places that volumes are created during that test:

https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_clone.py#L52,L55

uses the common create_volume helper class which always registers the volumes for cleanup at the end of the test class:

https://github.com/openstack/tempest/blob/master/tempest/api/volume/base.py#L133

which gets used during clear_volumes():

https://github.com/openstack/tempest/blob/master/tempest/api/volume/base.py#L187-L198

which is called by tearDownClass():

https://github.com/openstack/tempest/blob/master/tempest/api/volume/base.py#L109

Also it's worth point out this teardown mechanism is shared with all the volume api tests for the most part.

Do you have a link to the log (or attach a tempest or cinder api log) where the it shows tempest not making the appropriate volume delete calls ? I don't see where the teardown leak is coming from

[Lucio Seki (lseki)]

This Hitachi HNAS NFS log shows that volume volume-4d149723-7eb0-46df-a3c3-87eda159c2f1 was not deleted:

http://openstack.fit-tecnologia.org.br:10000/58/404958/27/check/hitachi-hnas-nfs/6bf4c8e/logs/screen-c-vol.txt.gz

2017-01-02 14:34:57.873 INFO cinder.volume.manager [req-a3cc877d-2f43-4bd6-9cd7-b6e197820dc5 None] [volume-4d149723-7eb0-46df-a3c3-87eda159c2f1] Created volume successfully.

---

And this EMC VMAX FC log shows that CI-5284794c-bde4-4c66-a178-e4ae309b569d was not deleted:

http://publiclogs.emc.com/vmax_ostack/EMC_VMAX_FC/338/338/logs/screen-c-vol.txt.gz

2017-01-02 22:04:11.779 INFO cinder.volume.manager [req-cd3c4607-28e6-4c93-a874-b42d6f02d569 None] [CI-5284794c-bde4-4c66-a178-e4ae309b569d] Created volume successfully.

[Jordan Pittier (jordan-pittier)]

That's a bug in your driver. The volume created from 8f06132a-ea9d is ee1ade38-0f1b and it is deleted. If your backend creates one or several intermediary cinder volumes to support the clone feature, there's nothing Tempest can do.

[Launchpad Janitor (janitor)]

[Expired for tempest because there has been no activity for 60 days.]