GlusterFS CI for Cinder fails for encrypted volumes tests

[Deepak C Shetty]

Here is an example:

==============================
2015-07-10 06:18:58.777 | Failed 2 tests - output below:
2015-07-10 06:18:58.777 | ==============================
2015-07-10 06:18:58.777 |
2015-07-10 06:18:58.777 | tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes.test_encrypted_cinder_volumes_cryptsetup[compute,id-cbc752ed-b716-4717-910f-956cce965722,image,volume]
2015-07-10 06:18:58.777 | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2015-07-10 06:18:58.777 |
2015-07-10 06:18:58.777 | Captured traceback:
2015-07-10 06:18:58.777 | ~~~~~~~~~~~~~~~~~~~
2015-07-10 06:18:58.778 | Traceback (most recent call last):
2015-07-10 06:18:58.778 | File "tempest/test.py", line 126, in wrapper
2015-07-10 06:18:58.778 | return f(self, *func_args, **func_kwargs)
2015-07-10 06:18:58.778 | File "tempest/scenario/test_encrypted_cinder_volumes.py", line 76, in test_encrypted_cinder_volumes_cryptsetup
2015-07-10 06:18:58.778 | self.attach_detach_volume()
2015-07-10 06:18:58.778 | File "tempest/scenario/test_encrypted_cinder_volumes.py", line 57, in attach_detach_volume
2015-07-10 06:18:58.778 | self.nova_volume_attach()
2015-07-10 06:18:58.778 | File "tempest/scenario/manager.py", line 420, in nova_volume_attach
2015-07-10 06:18:58.778 | self.volumes_client.wait_for_volume_status(volume['id'], 'in-use')
2015-07-10 06:18:58.778 | File "tempest/services/volume/json/volumes_client.py", line 182, in wait_for_volume_status
2015-07-10 06:18:58.778 | raise exceptions.TimeoutException(message)
2015-07-10 06:18:58.778 | tempest.exceptions.TimeoutException: Request timed out
2015-07-10 06:18:58.779 | Details: Volume 6052619b-c53c-4747-87c5-318b7e2aedf9 failed to reach in-use status (current: available) within the required time (196 s).
2015-07-10 06:18:58.779 |

 tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes.test_encrypted_cinder_volumes_luks[compute,id-79165fb4-5534-4b9d-8429-97ccffb8f86e,image,volume]
2015-07-10 06:18:59.066 | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2015-07-10 06:18:59.066 |
2015-07-10 06:18:59.066 | Captured traceback:
2015-07-10 06:18:59.066 | ~~~~~~~~~~~~~~~~~~~
2015-07-10 06:18:59.066 | Traceback (most recent call last):
2015-07-10 06:18:59.066 | File "tempest/test.py", line 126, in wrapper
2015-07-10 06:18:59.067 | return f(self, *func_args, **func_kwargs)
2015-07-10 06:18:59.067 | File "tempest/scenario/test_encrypted_cinder_volumes.py", line 67, in test_encrypted_cinder_volumes_luks
2015-07-10 06:18:59.067 | self.attach_detach_volume()
2015-07-10 06:18:59.067 | File "tempest/scenario/test_encrypted_cinder_volumes.py", line 57, in attach_detach_volume
2015-07-10 06:18:59.067 | self.nova_volume_attach()
2015-07-10 06:18:59.067 | File "tempest/scenario/manager.py", line 420, in nova_volume_attach
2015-07-10 06:18:59.067 | self.volumes_client.wait_for_volume_status(volume['id'], 'in-use')
2015-07-10 06:18:59.067 | File "tempest/services/volume/json/volumes_client.py", line 182, in wait_for_volume_status
2015-07-10 06:18:59.067 | raise exceptions.TimeoutException(message)
2015-07-10 06:18:59.067 | tempest.exceptions.TimeoutException: Request timed out
2015-07-10 06:18:59.067 | Details: Volume ae6bb02b-e271-4945-8bc2-330f9a5e9a18 failed to reach in-use status (current: available) within the required time (196 s).

[Deepak C Shetty (dpkshetty)]

Additional Info to set some context :

TestEncryptedCinderVolumes testcase(s) of tempest won't be supported by GlusterFS backend until nova volume driver has ability to understand and work with GlusterFS volumes. The same issue is with Ceph too. After recent changes to fix a false positive bug where this testcase was incorrectly being passed, most volume drivers will fail, unless they have support in Nova for handlign encryption for that backend. For more details:

* list email that explains the issue

  * http://lists.openstack.org/pipermail/openstack-dev/2015-June/068117.html

* Cinder patch to enable 'encryption' for all volume drivers (if specified by admin/user)

  * https://review.openstack.org/#/c/193673/

  * NOTE: GlusterFS CI job fails here and is expected as nova fails to encrypt hence attach the volume

  * See http://logs.openstack.org/73/193673/3/check/check-tempest-dsvm-full-glusterfs-nv/1d497c8/console.html#_2015-06-29_08_25_14_612

* Nova fix to handle the case where cinder volume driver asks for encryption but Nova doesn't have support for it

  * https://review.openstack.org/#/c/193830/

* Tempest new config option to skip encrypted volume tests

  * https://review.openstack.org/#/c/193831/

[Deepak C Shetty (dpkshetty)]

Fix is to disable encrypted volume tests for GlusterFS backend
Patch for same posted @

https://review.openstack.org/#/c/200399/

The glusterfs CI job is passing for the above patch.

From the CI logs:

2015-07-10 07:44:18.123 | {1} setUpClass (tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes) ... SKIPPED: Encrypted volume attach is not supported

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/200399
Committed: https://git.openstack.org/cgit/stackforge/devstack-plugin-glusterfs/commit/?id=fb470e7c6cdf9b57b5698f7cf73cb5ed70c18111
Submitter: Jenkins
Branch: master

commit fb470e7c6cdf9b57b5698f7cf73cb5ed70c18111
Author: Deepak C Shetty <email address hidden>
Date: Fri Jul 10 12:27:03 2015 +0530

    Skip encrypted volume tests

    The encrypted Cinder volume tests in Tempest don't actually work
    properly for a GlusterFS backend in cinder since the volume encryption
    support is not in Nova for GlusterFS backed volumes.

    Closes-Bug: 1473363
    Change-Id: I286d680256b0bd26d66d14c2554854f89abac7a8