OpenStack Shared File Systems Service (Manila)

glusterfs_driver: Allow access with NFS volume mapped layout is broken

Bug #1498835 reported by karthick on 2015-09-23

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Shared File Systems Service (Manila)	Fix Released	Medium	Csaba Henk	OpenStack Shared File Systems Service (Manila) 1.0.0 "liberty"

Bug Description

After addition of commit 2153b5fa473af9e8bd207935eccd8dc47d7ad07a to liberty master branch, attempting to allow access to a volume mapped layout fails with error - 'AttributeError: 'NoneType' object has no attribute 'split''

This issue is seen as export vol list is empty for a new volume created and we try to split an empty list.

    def _get_vol_exports(self):
        export_vol = self.gluster_manager.get_gluster_vol_option(
            NFS_RPC_AUTH_ALLOW)
        return export_vol.split(',')

snippet of logs:

2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 110, in wrapped
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher return f(self, *args, **kwargs)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 1066, in allow_access
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher context, access_mapping['id'], access_mapping.STATE_ERROR)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/manager.py", line 1060, in allow_access
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher share_server=share_server)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/glusterfs/layout.py", line 101, in allow_access
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher access, share_server)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/glusterfs/__init__.py", line 148, in _allow_access_via_manager
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher self._get_helper(gluster_mgr).allow_access('/', share, access)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/glusterfs/__init__.py", line 317, in allow_access
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher self._manage_access(access['access_type'], access['access_to'], cbk)
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/glusterfs/__init__.py", line 290, in _manage_access
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher export_vol_list = self._get_vol_exports()
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/manila/manila/share/drivers/glusterfs/__init__.py", line 266, in _get_vol_exports
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher return export_vol.split(',')
2015-09-23 12:34:24.39 TRACE oslo_messaging.rpc.dispatcher AttributeError: 'NoneType' object has no attribute 'split'

Tags:

Valeriy Ponomaryov (vponomaryov) on 2015-09-23

tags:

added: driver glusterfs

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-23: Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/226800

Changed in manila:
assignee:	nobody → Csaba Henk (chenk)
status:	New → In Progress

Csaba Henk (chenk) on 2015-09-24

tags:

added: liberty-rc-potential

Ben Swartzlander (bswartz) on 2015-09-28

Changed in manila:
importance:	Undecided → Medium

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-30: Fix merged to manila (master)

Reviewed: https://review.openstack.org/226800
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=42eb36b74594b6431c5cb0915a1bd327aca3f11f
Submitter: Jenkins
Branch: master

commit 42eb36b74594b6431c5cb0915a1bd327aca3f11f
Author: Csaba Henk <email address hidden>
Date: Wed Sep 23 15:56:22 2015 +0200

glusterfs: manage nfs.rpc-auth-allow not being set

    GlusterNFSVolHelper._get_vol_exports should return an
    empty list if nfs.rpc-auth-allow is not set on the
    backing volume.

Change-Id: I481bcef66efb7f49da7ea9dab27300825388d6d8
Closes-Bug: #1498835

Changed in manila:
status:	In Progress → Fix Committed

Ben Swartzlander (bswartz) on 2015-09-30

Changed in manila:
milestone:	none → liberty-rc2

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-01: Fix proposed to manila (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/229770

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-01: Fix merged to manila (stable/liberty)

Reviewed: https://review.openstack.org/229770
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=722a22eac20109081662cc1796560963578575c6
Submitter: Jenkins
Branch: stable/liberty

commit 722a22eac20109081662cc1796560963578575c6
Author: Csaba Henk <email address hidden>
Date: Wed Sep 23 15:56:22 2015 +0200

glusterfs: manage nfs.rpc-auth-allow not being set

    GlusterNFSVolHelper._get_vol_exports should return an
    empty list if nfs.rpc-auth-allow is not set on the
    backing volume.

    Change-Id: I481bcef66efb7f49da7ea9dab27300825388d6d8
    Closes-Bug: #1498835
    (cherry picked from commit 42eb36b74594b6431c5cb0915a1bd327aca3f11f)

tags:

added: in-stable-liberty

Clinton Knight (clintonk) on 2015-10-01

Changed in manila:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2015-10-15

Changed in manila:
milestone:	liberty-rc2 → 1.0.0

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-15: Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/235328

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-20: Fix merged to manila (master)

Download full text (11.9 KiB)

Reviewed: https://review.openstack.org/235328
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=ec94d6929ea8e1b4ce10dc91cd4954ece808668c
Submitter: Jenkins
Branch: master

commit f1eded1fbcaf309e1c9a4be3f8a14bd25daa3e46
Author: Gaurang Tapase <email address hidden>
Date: Mon Oct 12 20:47:19 2015 +0530

Fix usage of dependencies

Manila is broken is threee places, so fix them:
1) test 'test_misc' with WebOb 1.5

    WebOb 1.5 was released at 2015-10-11. With this new version,
    webob.exc.WSGIHTTPException() constructor now fails with a KeyError
    when the HTTP status code is 0.

    test_exceptions_raise() of test_misc tries to instantiate all
    exceptions of manila.exception. The problem is that
    ConvertedException uses a default HTTP status code of 0.

Modify the default HTTP status code of ConvertedException to 400 to
fix the unit tests.

2) Add dependency for 'testresources' that is required by migration
tests.

    3) Remove 2 unit tests related to testing of oslo.policy lib
    functionality that should not be tested in Manila. It started failing
    because under-the-hood behaviour was changed in new realese 0.12.0

Closes-Bug: #1505153
Closes-Bug: #1505374

(cherry picked from commit 9c99814ce5943bd4c33bf3650b832666e31b3411)

-- squashed with another change to get tests to pass on stable/liberty --

Fix broken unit tests

With release of six module version 1.10.0 several our unit tests
started to fail because of usage of not strict constructions.

    Changes:
    1) Manila unit test
    "manila.tests.share.test_api.ShareAPITestCase.test_extend_quota_error"
    used str for int substitution. So, use int data for int substitution.

    2) Module 'manila.share.drivers.hp.hp_3par_mediator' was using
    LOG.exception function when no traceback were exist it led to
    AttributeError on py34. So, replace all usages of 'LOG.exception'
    with 'LOG.error' where no raised exceptions exist.

    Change-Id: Ic5b37bfb9d939d03f6ff68bc53d134bf9e5f996e
    Closes-Bug: #1503969
    (cherry picked from commit f38b8d4efd1f68f4ea29747f7377e0936f61d89c)

Change-Id: I0f28f3c3fb2c7eec1bafc3a617344990f86810cf

commit 151b691bb2d4baa436913924df60b8c197f91463
Author: Valeriy Ponomaryov <email address hidden>
Date: Thu Oct 1 12:42:05 2015 +0300

Fix display of availability-zone for manila-manage command

Commands "manila-manage service list" and "manila-manage host list"
were displaying availability zone instance instead of its name.

Such bug appeared after implementation of Availability zone model.
So, fix it by providing 'name' field of availability zone model.

Change-Id: I14c3451380df01853183aed265344b1783c95939
Closes-Bug: #1499677

commit 77455a5ac6be828e8dfd3e75566eaff2823595d4
Author: Csaba Henk <email address hidden>
Date: Wed Sep 30 14:57:00 2015 +0200

glusterfs_native: use dynamic-auth option if available

With dynamic-auth restarting the volume is not necessary
in deny_access.

Change-Id: Ic25af1795c279b34370...

Reviewed:  https://review.openstack.org/235328
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=ec94d6929ea8e1b4ce10dc91cd4954ece808668c
Submitter: Jenkins
Branch:    master

commit f1eded1fbcaf309e1c9a4be3f8a14bd25daa3e46
Author: Gaurang Tapase <gaurang.tapase@in.ibm.com>
Date:   Mon Oct 12 20:47:19 2015 +0530

Fix usage of dependencies
    
    Manila is broken is threee places, so fix them:
    1) test 'test_misc' with WebOb 1.5
    
    WebOb 1.5 was released at 2015-10-11. With this new version,
    webob.exc.WSGIHTTPException() constructor now fails with a KeyError
    when the HTTP status code is 0.
    
    test_exceptions_raise() of test_misc tries to instantiate all
    exceptions of manila.exception. The problem is that
    ConvertedException uses a default HTTP status code of 0.
    
    Modify the default HTTP status code of ConvertedException to 400 to
    fix the unit tests.
    
    2) Add dependency for 'testresources' that is required by migration
    tests.
    
    3) Remove 2 unit tests related to testing of oslo.policy lib
    functionality that should not be tested in Manila. It started failing
    because under-the-hood behaviour was changed in new realese 0.12.0
    
    Closes-Bug: #1505153
    Closes-Bug: #1505374
    
    (cherry picked from commit 9c99814ce5943bd4c33bf3650b832666e31b3411)
    
    -- squashed with another change to get tests to pass on stable/liberty --
    
    Fix broken unit tests
    
    With release of six module version 1.10.0 several our unit tests
    started to fail because of usage of not strict constructions.
    
    Changes:
    1) Manila unit test
    "manila.tests.share.test_api.ShareAPITestCase.test_extend_quota_error"
    used str for int substitution. So, use int data for int substitution.
    
    2) Module 'manila.share.drivers.hp.hp_3par_mediator' was using
    LOG.exception function when no traceback were exist it led to
    AttributeError on py34. So, replace all usages of 'LOG.exception'
    with 'LOG.error' where no raised exceptions exist.
    
    Change-Id: Ic5b37bfb9d939d03f6ff68bc53d134bf9e5f996e
    Closes-Bug: #1503969
    (cherry picked from commit f38b8d4efd1f68f4ea29747f7377e0936f61d89c)
    
    --
    
    Change-Id: I0f28f3c3fb2c7eec1bafc3a617344990f86810cf

commit 151b691bb2d4baa436913924df60b8c197f91463
Author: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Date:   Thu Oct 1 12:42:05 2015 +0300

Fix display of availability-zone for manila-manage command
    
    Commands "manila-manage service list" and "manila-manage host list"
    were displaying availability zone instance instead of its name.
    
    Such bug appeared after implementation of Availability zone model.
    So, fix it by providing 'name' field of availability zone model.
    
    Change-Id: I14c3451380df01853183aed265344b1783c95939
    Closes-Bug: #1499677

commit 77455a5ac6be828e8dfd3e75566eaff2823595d4
Author: Csaba Henk <chenk@redhat.com>
Date:   Wed Sep 30 14:57:00 2015 +0200

glusterfs_native: use dynamic-auth option if available
    
    With dynamic-auth restarting the volume is not necessary
    in deny_access.
    
    Change-Id: Ic25af1795c279b34370e25ebdc80c2853a76cc4b
    Closes-bug: #1489081
    (cherry picked from commit 74cc46c76b5c18481726768edc4a79e9e5a7c706)

commit 2a6c8d71c97d1d70d3fd3eb7ba3e43a2f92e9a5f
Author: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Date:   Wed Sep 23 13:54:00 2015 +0300

Fix setting of "snapshot_support" extra spec for tempest
    
    Tempest test module "test_shares_actions.py" uses custom share_type,
    but do not allow to redefine extra spec "snapshot_support".
    And fails for drivers that do not have snapshot support and report
    such capability as "False".
    
    Changes:
    - Add new config option called "capability_snapshot_support" that
    will be used for each share type created in Tempest by default.
    - Make it default to existing config option "run_snapshot_tests"
    as they will be equal in most cases. But separate their logic, as
    we may want just to disable snapshot tests running tempest locally
    and testing some other feature having snapshot support in back end.
    - Rename existing config option "storage_protocol" to
    "capability_storage_protocol" for consistency with new option. And
    keep old name as "deprecated".
    
    Change-Id: I9ba0a9b10ffc3f0fda6094a3f5cad26a2e8a447f
    Closes-Bug: #1498858
    (cherry picked from commit 29f2695eb9c615d258344bbc2a835e98940c6290)

commit b3256350b889b5c5522d73c2bb7e3d27ebfe27f0
Author: Julia Varlamova <jvarlamova@mirantis.com>
Date:   Tue Sep 29 06:09:05 2015 -0400

Fix deletion of error state access rules
    
    Deny access to share instance in case access rule is in error state
    and has access to share instances.
    
    Add function 'share_instance_access_get_all' to DB API to check if we can
    delete share access or need to remove access to share instance at first.
    
    Closes-bug: #1495892
    
    Change-Id: Id1f138e6bb48daca0a768dea7440829185fca9e5
    (cherry picked from commit ccb3a1c5874dfdbfe6192b6acadacb97a050b1bd)

commit 4d9f87d6ccf30b2fb2c6f34d2a25d93717131461
Author: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Date:   Wed Sep 30 12:07:50 2015 +0300

Fix response data for API access-allow
    
    API access-allow returns following redundant data:
    - deleted
    - deleted_at
    - instance_mappings
    
    So, remove them for create share api method and cover it with
    unit and tempest tests.
    
    Change-Id: I103380c3f0008d4879b39f339187e57eeec40456
    Closes-Bug: #1499733
    (cherry picked from commit beb9191c69058cecb3c961297213a1c59f65745e)

commit 5671c464f75560abf979eb1ca805a4da32373089
Author: Csaba Henk <chenk@redhat.com>
Date:   Thu Sep 24 03:03:54 2015 +0200

glusterfs: check nfs.export-volumes with Gluster NFS + vol layout
    
    nfs.export-volumes is required to be set to 'on' on the backing
    GlusterFS cluster (nb. it's a cluster-wide setting) if the driver's
    setup is
    
        glusterfs_nfs_server_type = Gluster
        glusterfs_share_layout = layout_volume.GlusterfsVolumeMappedLayout
    
    Change-Id: I472eb5534110e8b216275ad3f4295f27a81f9815
    Closes-Bug: #1499124
    (cherry picked from commit d7ff0f43148c03757bde1b009c5b6b00611cefa8)

commit 722a22eac20109081662cc1796560963578575c6
Author: Csaba Henk <chenk@redhat.com>
Date:   Wed Sep 23 15:56:22 2015 +0200

glusterfs: manage nfs.rpc-auth-allow not being set
    
    GlusterNFSVolHelper._get_vol_exports should return an
    empty list if nfs.rpc-auth-allow is not set on the
    backing volume.
    
    Change-Id: I481bcef66efb7f49da7ea9dab27300825388d6d8
    Closes-Bug: #1498835
    (cherry picked from commit 42eb36b74594b6431c5cb0915a1bd327aca3f11f)

commit be76d9e404c0f8a9728bccb7b54dffae091d0700
Author: Csaba Henk <chenk@redhat.com>
Date:   Tue Sep 29 09:49:19 2015 +0200

glusterfs vol layout: start volume cloned from snapshot
    
    When handling create_share_from_snapshot with glusterfs
    volume layout, we do a snapshot clone gluster operation
    that gives us a new volume (which will be used to back
    the new share). 'snapshot clone' does not start the
    resultant volume, we have explicitly start it from Manila.
    So far the volume layout code did not bother about it,
    rather the 'vol start' was called from glusterfs-native
    driver. That however broke all other volume layout based
    configs (ie. glusterfs driver with vol layout).
    
    Fix this now by doing the 'vol start' call in the vol
    layout code.
    
    Change-Id: I63c13ce468a3227f09e381814f55e8c914fbef95
    Closes-Bug: #1499347
    (cherry picked from commit 4e4c8759a25fa18384b45f4150000aacfaed035a)

commit ae378ce27d961f074953eccae0047bdf5af8821f
Author: Clinton Knight <cknight@netapp.com>
Date:   Sun Sep 27 15:32:05 2015 -0700

NetApp cDOT driver isn't reentrant
    
    While testing the cDOT driver at high concurrency (12), we observed a few
    occasional errors that were difficult to explain until we noticed that
    some APIs were being sent to the wrong vserver. Root cause is that the
    protocol helpers are cached and reused despite being stateful. There is
    virtually no performance benefit to caching the helper classes, so the
    simplest fix is to stop caching them.
    
    Change-Id: I30efcf5a3ada4cc03d8506738876498ae75d20d3
    Closes-Bug: #1500257
    (cherry picked from commit 7049a9aac9550c449deefcd129aa871924d013c8)

commit 91b88b2167a239f93df5cf41b993261abbc0f4d7
Author: Clinton Knight <cknight@netapp.com>
Date:   Wed Sep 23 10:48:34 2015 -0400

Can't create shares on drivers that don't support snapshots
    
    For a driver that doesn't support snapshots, a share type with
    snapshot_support = '<is> False' will prevent share creation.
    Root cause is code in HostManager that improperly checks whether
    snapshot_support (and other boolean properties) is set on each
    pool. This commit fixes the check for the boolean pool capabilities.
    
    Change-Id: I6a488788926cca119498b3de992beec3b0545259
    Closes-Bug: #1498646
    (cherry picked from commit 8a34fb3890810330a9b8a2d659739d92134a832c)

commit b28c8fccf93ef2adb08685e3efa1b5d72b0c7f94
Author: Clinton Knight <cknight@netapp.com>
Date:   Thu Sep 24 17:27:04 2015 -0400

Revert netapp_lib dependency in NetApp cDOT Manila drivers
    
    The community asked us to remove the dependency on a PyPI
    library with a proprietary license.  This commit does that
    while preserving some minor refactoring that was done at the
    same time.
    
    Reverts commit ca6168d01bea405718bbf6a36d4bb75c23d1b783.
    
    Closes-Bug: #1499334
    Change-Id: I1a4b1b2b2151bb222ae5c77b9b317e7505255067
    (cherry picked from commit f925c049007fbd5ec71a4ec59de1120d35ad66af)

commit 1b020560e11aa048d8f0da4f755fe3ad1cbdc0f5
Author: Ben Swartzlander <ben@swartzlander.org>
Date:   Wed Sep 30 15:55:02 2015 -0400

Set defaultbranch to stable/liberty in .gitreview
    
    Change-Id: I67a1cd670f18822f5918537fda8b451a9d72088c

commit 552a80a5bfa48e5c53e80c5fef10e08bfe62f614
Author: Nicolas Trangez <ikke@nicolast.be>
Date:   Fri Sep 25 15:26:57 2015 +0200

Fix `test_trans_add` for Python 3.4.3
    
    The `test_trans_add` test in `manila.tests.test_hacking.HackingTestCase`
    consistently failed on our CI system, which is based on Ubuntu 14.04.
    Contrary, on my development machine running Fedora 22, all tests
    succeeded. Whilst the test dispatches on Python version (`if six.PY2`)
    to select which `hacking` errors are expected, at which line/column
    pair, I noticed our CI system returned the expected values for Python 2,
    not the Python 3 ones, which differ in column numbers only.
    
    After verifying versions of dependencies, stepping through the code and
    whatnot, the only difference left turned out to be the version of Python
    being used: 3.4.2 on my workstation, 3.4.3 on our CI system. As a last
    resort, I opened the Python 3.4 ChangeLog [1] and noticed a suspicious
    entry::
    
        Issue #21295: Revert some changes (issue #16795) to AST line numbers
        and column offsets that constituted a regression.
    
    Looking at those issues, it becomes clear this is the cause. Supposedly
    the Python 3 specific expected values were created on a Python 3.4
    version containing the original patch of #16795 [2], and this is also
    what's running on the OpenStack CI system. Our CI system runs a build of
    Python which contains the revert of #21295 [3].
    
    This patch fixes the version-specific expected error calculation by not
    simply dispatching on Python 2 or 3, but specifically limits the custom
    version to 3.4.0 <= Python < 3.4.3.
    
    [1] https://docs.python.org/3.4/whatsnew/changelog.html
    [2] http://bugs.python.org/issue16795
    [3] http://bugs.python.org/issue21295
    
    Closes-Bug: 1499743
    
    Change-Id: I649fb1f5244efba7ab79e9bf337433d541fa8b19
    (cherry picked from commit 83167bd2a2a0369d2271bd08ebc7ca753a603d60)

Revision history for this message

Doug Hellmann (doug-hellmann) wrote on 2015-12-03: Fix included in openstack/manila 2.0.0.0b1

This issue was fixed in the openstack/manila 2.0.0.0b1 development milestone.