Dell powermax/vnx/unity manila driver error: 'Share network security service association is mandatory for protocol CIFS'

[Sam Wan]

SharesCIFSTest failed recently on Dell powermax/vnx/unity manila drivers.

Below is vnx example
====================================
2021-08-16 05:12:14.084 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_delete_snapshot [23.008231s] ... FAILED
2021-08-16 05:12:25.924 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_get_delete_share [11.833495s] ... FAILED
2021-08-16 05:12:47.735 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_share_from_snapshot [21.805745s] ... FAILED
2021-08-16 05:13:09.719 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_share_from_snapshot_share_network_not_provided [21.978814s] ... FAILED
======================================

check manila share log
=====================================
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server [None req-5ad58f51-cc62-4188-be93-0692daefb8ff demo None] Exception during message handling: manila.exception.InvalidRequest: Share network security service association is mandatory for protocol CIFS.
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/manila/manila/share/manager.py", line 216, in wrapped
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/manila/manila/utils.py", line 578, in wrapper
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/manila/manila/share/manager.py", line 2034, in create_share_instance
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server raise exception.InvalidRequest(_(
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server manila.exception.InvalidRequest: Share network security service association is mandatory for protocol CIFS.
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server
===================

The error is related to 776875: Early validate for CIFS without security service. | https://review.opendev.org/c/openstack/manila/+/776875

attached manila share log and tempest log

[Goutham Pacha Ravi (gouthamr)]

Hi Sam,

Don't VNX, Powermax and Unity drivers *need* an AD configured for CIFS shares to work?

I see the following in your tempest configuration:

share.share_network_id = 84a42a13-04a6-4f4a-ad8e-2339e0c69921

Does this share network have an active_directory security service created?

[Sam Wan (sam-wan)]

Hi Goutham,

Yes powermax/unity/vnx have and AD configure for CIFS shares.
Below is the configure for this VNX build:
===========================================
2021-08-16 04:46:49.402 | + environment:prepare_share_network_and_security_service:17 : manila security-service-create --dns-ip 52.0.0.254 --domain vnxci.elab --server vnxadsvr --user Administrator --password password --name nas_ad active_directory
...
2021-08-16 04:46:51.291 | + environment:prepare_share_network_and_security_service:19 : manila share-network-create --neutron-net-id d4f64258-147f-4cea-96b5-c02bfa3d2647 --neutron-subnet-id 6aedff1d-e8d6-42cd-9de6-f83f6551d9ec --name nas_sharenet
...
2021-08-16 04:46:54.905 | + environment:prepare_share_network_and_security_service:21 : manila share-network-security-service-add nas_sharenet nas_ad
===========================================

security service config as below
===========================================
$ manila security-service-show nas_ad
+----------------+------------------------------------------+
| Property | Value |
+----------------+------------------------------------------+
| id | 74e0cadf-bc9d-47e8-845b-605e94590c48 |
| name | nas_ad |
| type | active_directory |
| status | new |
| created_at | 2021-08-16T08:46:51.002318 |
| updated_at | None |
| description | None |
| dns_ip | 52.0.0.254 |
| server | vnxadsvr |
| domain | vnxci.elab |
| user | Administrator |
| password | password |
| project_id | d9ff5e00e53947a0b2a53f299bf16281 |
| ou | None |
| share_networks | ['84a42a13-04a6-4f4a-ad8e-2339e0c69921'] |
+----------------+------------------------------------------+
===========================================

You can see that the share network has an AD security service.

Our ci has no issue before the change was merged.

thanks and regards
Sam

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/manila/+/804814

[Sam Wan (sam-wan)]

Hi Goutham,

I've known the cause.
The error happens here:
===============================
        if share_network_id and self.driver.driver_handles_share_servers:
            proto = share_instance.get('share_proto').lower()
            ret_types = (
                self.driver.dhss_mandatory_security_service_association.get(
                    proto))
            if ret_types:
                share_network = self.db.share_network_get(context,
                                                          share_network_id)
                share_network_ss = []
                for security_service in share_network['security_services']:
                    share_network_ss.append(security_service['type'].lower())
                for types in ret_types: #<--- error
                    if types not in share_network_ss:
                        self.db.share_instance_update(
                            context, share_instance_id,
                            {'status': constants.STATUS_ERROR}
                        )
...
===============================

For Dell manila drivers, the value of 'cifs' is a string instead of a list
===============================
--
powermax/connection.py: self.dhss_mandatory_security_service_association = {
powermax/connection.py- 'nfs': None,
powermax/connection.py- 'cifs': 'active_directory',
--
unity/connection.py: self.dhss_mandatory_security_service_association = {
unity/connection.py- 'nfs': None,
unity/connection.py- 'cifs': 'active_directory',
--
vnx/connection.py: self.dhss_mandatory_security_service_association = {
vnx/connection.py- 'nfs': None,
vnx/connection.py- 'cifs': 'active_directory',
===============================

Thus 'for types in ret_types' does not work as expected. types is a letter instead of a valid security service type name.

The fix is simple and I've submitted a change for it.
https://review.opendev.org/c/openstack/manila/+/804814
Please help to review.

thanks and regards

Sam

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/manila/+/804814
Committed: https://opendev.org/openstack/manila/commit/5bd44e0be35a8710dd0e5ad9c35f924b0199e971
Submitter: "Zuul (22348)"
Branch: master

commit 5bd44e0be35a8710dd0e5ad9c35f924b0199e971
Author: Sam Wan <email address hidden>
Date: Tue Aug 17 12:01:44 2021 +0800

    Change cifs value from string to list for Dell manila drivers

    Dell manila drivers use string as value for 'cifs' in
    dhss_mandatory_security_service_association.
    This fix changes it to a list.

    Change-Id: I0c64e574301baf2a41a475af3b3848cbec8d495f
    Closes-Bug: #1940072

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 13.0.0.0rc1 release candidate.