Dell powermax/vnx/unity manila driver error: 'Share network security service association is mandatory for protocol CIFS'

Bug #1940072 reported by Sam Wan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Undecided
Unassigned

Bug Description

SharesCIFSTest failed recently on Dell powermax/vnx/unity manila drivers.

Below is vnx example
====================================
2021-08-16 05:12:14.084 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_delete_snapshot [23.008231s] ... FAILED
2021-08-16 05:12:25.924 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_get_delete_share [11.833495s] ... FAILED
2021-08-16 05:12:47.735 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_share_from_snapshot [21.805745s] ... FAILED
2021-08-16 05:13:09.719 | {0} manila_tempest_tests.tests.api.test_shares.SharesCIFSTest.test_create_share_from_snapshot_share_network_not_provided [21.978814s] ... FAILED
======================================

check manila share log
=====================================
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server [None req-5ad58f51-cc62-4188-be93-0692daefb8ff demo None] Exception during message handling: manila.exception.InvalidRequest: Share network security service association is mandatory for protocol CIFS.
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/manila/manila/share/manager.py", line 216, in wrapped
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/manila/manila/utils.py", line 578, in wrapper
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs)
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server File "/opt/stack/new/manila/manila/share/manager.py", line 2034, in create_share_instance
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server raise exception.InvalidRequest(_(
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server manila.exception.InvalidRequest: Share network security service association is mandatory for protocol CIFS.
Aug 16 05:11:53.016771 e2e-os-vnxmanila190 manila-share[142336]: ERROR oslo_messaging.rpc.server
===================

The error is related to 776875: Early validate for CIFS without security service. | https://review.opendev.org/c/openstack/manila/+/776875

attached manila share log and tempest log

Revision history for this message
Sam Wan (sam-wan) wrote :
tags: added: dell-emc powermax unity vnx
Revision history for this message
Sam Wan (sam-wan) wrote :
Revision history for this message
Goutham Pacha Ravi (gouthamr) wrote :

Hi Sam,

Don't VNX, Powermax and Unity drivers *need* an AD configured for CIFS shares to work?

I see the following in your tempest configuration:

share.share_network_id = 84a42a13-04a6-4f4a-ad8e-2339e0c69921

Does this share network have an active_directory security service created?

Revision history for this message
Sam Wan (sam-wan) wrote :

Hi Goutham,

Yes powermax/unity/vnx have and AD configure for CIFS shares.
Below is the configure for this VNX build:
===========================================
2021-08-16 04:46:49.402 | + environment:prepare_share_network_and_security_service:17 : manila security-service-create --dns-ip 52.0.0.254 --domain vnxci.elab --server vnxadsvr --user Administrator --password password --name nas_ad active_directory
...
2021-08-16 04:46:51.291 | + environment:prepare_share_network_and_security_service:19 : manila share-network-create --neutron-net-id d4f64258-147f-4cea-96b5-c02bfa3d2647 --neutron-subnet-id 6aedff1d-e8d6-42cd-9de6-f83f6551d9ec --name nas_sharenet
...
2021-08-16 04:46:54.905 | + environment:prepare_share_network_and_security_service:21 : manila share-network-security-service-add nas_sharenet nas_ad
===========================================

security service config as below
===========================================
$ manila security-service-show nas_ad
+----------------+------------------------------------------+
| Property | Value |
+----------------+------------------------------------------+
| id | 74e0cadf-bc9d-47e8-845b-605e94590c48 |
| name | nas_ad |
| type | active_directory |
| status | new |
| created_at | 2021-08-16T08:46:51.002318 |
| updated_at | None |
| description | None |
| dns_ip | 52.0.0.254 |
| server | vnxadsvr |
| domain | vnxci.elab |
| user | Administrator |
| password | password |
| project_id | d9ff5e00e53947a0b2a53f299bf16281 |
| ou | None |
| share_networks | ['84a42a13-04a6-4f4a-ad8e-2339e0c69921'] |
+----------------+------------------------------------------+
===========================================

You can see that the share network has an AD security service.

Our ci has no issue before the change was merged.

thanks and regards
Sam

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/manila/+/804814

Changed in manila:
status: New → In Progress
Revision history for this message
Sam Wan (sam-wan) wrote :

Hi Goutham,

I've known the cause.
The error happens here:
===============================
        if share_network_id and self.driver.driver_handles_share_servers:
            proto = share_instance.get('share_proto').lower()
            ret_types = (
                self.driver.dhss_mandatory_security_service_association.get(
                    proto))
            if ret_types:
                share_network = self.db.share_network_get(context,
                                                          share_network_id)
                share_network_ss = []
                for security_service in share_network['security_services']:
                    share_network_ss.append(security_service['type'].lower())
                for types in ret_types: #<--- error
                    if types not in share_network_ss:
                        self.db.share_instance_update(
                            context, share_instance_id,
                            {'status': constants.STATUS_ERROR}
                        )
...
===============================

For Dell manila drivers, the value of 'cifs' is a string instead of a list
===============================
--
powermax/connection.py: self.dhss_mandatory_security_service_association = {
powermax/connection.py- 'nfs': None,
powermax/connection.py- 'cifs': 'active_directory',
--
unity/connection.py: self.dhss_mandatory_security_service_association = {
unity/connection.py- 'nfs': None,
unity/connection.py- 'cifs': 'active_directory',
--
vnx/connection.py: self.dhss_mandatory_security_service_association = {
vnx/connection.py- 'nfs': None,
vnx/connection.py- 'cifs': 'active_directory',
===============================

Thus 'for types in ret_types' does not work as expected. types is a letter instead of a valid security service type name.

The fix is simple and I've submitted a change for it.
https://review.opendev.org/c/openstack/manila/+/804814
Please help to review.

thanks and regards

Sam

Vida Haririan (vhariria)
tags: added: ci
tags: added: cifs
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/c/openstack/manila/+/804814
Committed: https://opendev.org/openstack/manila/commit/5bd44e0be35a8710dd0e5ad9c35f924b0199e971
Submitter: "Zuul (22348)"
Branch: master

commit 5bd44e0be35a8710dd0e5ad9c35f924b0199e971
Author: Sam Wan <email address hidden>
Date: Tue Aug 17 12:01:44 2021 +0800

    Change cifs value from string to list for Dell manila drivers

    Dell manila drivers use string as value for 'cifs' in
    dhss_mandatory_security_service_association.
    This fix changes it to a list.

    Change-Id: I0c64e574301baf2a41a475af3b3848cbec8d495f
    Closes-Bug: #1940072

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 13.0.0.0rc1

This issue was fixed in the openstack/manila 13.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.