Volume encryption tests handling on multibackend environments

[Luigi Toscano]

When multiple volume backends are available, encryption tests may fail if encryption is not supported on some backends. For example, whenever the NFS protocol is used, encryption does not work (proof: there is work in progress here https://review.opendev.org/#/c/597148/).

As the environment as a whole supports encryption, It should be possible to run encryption tests but limit them on the supported backend(s).

I see various possible solutions:

1) make sure volume.storage_protocol is used when calling the volume type is create inside EncryptionScenarioTest.create_encrypted_volume

2) add a new configuration variable which will be used when calling EncryptionScenarioTest.create_encrypted_volume and then create_volume_type.

This has two possible variants:
2a) let this configuration key list a single backend. This may be limit the tested scenarios if more backend are capable of encryption;
2b) allow for a list of values, and choose one randomly. This may introduce too much randomness and it is not exactly the same as "allow cinder to choose any capable resource".

3) assume that the first backend in volume.backend_list work with encryption. This is a bit risky and potentially limits which scenarios can be tested.

[Martin Kopec (mkopec)]

What exactly is failing? I don't see any tempest relevant failures in that review, some cinder tests are failing there but that's plugin's issue, isn't it?

[Luigi Toscano (ltoscano)]

That review is just to say that the generic NFS backend does not support encryption, as the required changes are still work in progress. Please go through the rest of the bug. As far as I know, there are no jobs currently on zuul.openstack.org which tests cinder multibackend where one of the backends doesn't support encryption.

[Luigi Toscano (ltoscano)]

Hopefully the description is more clear now.

[Martin Kopec (mkopec)]

Makes sense now, thanks

[Luigi Toscano (ltoscano)]

Thanks, now mostly looking for suggestions on how to proceed (I can write the code when a solution is chosen).

[Ghanshyam Mann (ghanshyammann)]

It is difficult things to test in case of multi-backends. Encryption or any other backends specific features also has the same issue.

Tempest test the features without knowledge of backend, few tests are with feature flag so that those test can be disabled in env backends does not support that feature and few are asked to skip if feature is most common feature to implement by backends but not implemented yet. This is all good and what we agreed in past for Tempest to do.

But for multi-backends case, knowing what backend can support xyz(encryption volume) feature is difficult. All the three options tosky mentioned in this bug has some risk. But at the same time we do not have any generic solution yet.

we discussed this on IRC: http://eavesdrop.openstack.org/irclogs/%23openstack-qa/%23openstack-qa.2020-05-26.log.html#t2020-05-26T13:55:16

Let's discuss in Virtual PTG about best possible way for multi backends cases.
- https://etherpad.opendev.org/p/qa-victoria-ptg

[Benny Kopilov (bkopilov)]

I would recommend the next solution:
[volume]
backend_names = {
tripleo_iscsi: {storage_protocol = iSCSI, vendor_name = XXX}
tripleo_nfs:{storage_protocol = YYY, vendor_name = ZZZ}
}

Maybe we can set a dictionary for backend_names and store the info.

Benny