bare exceptions in ganesha manager

Bug #1779922 reported by Tom Barron
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Low
Tom Barron

Bug Description

The ganesha manager has several places that raise bare exceptions instead of passing on useful information like the actual error from a failed process execution. Additionally, they produce an ugly traceback from oslo_messaging, for example:

Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 ERROR oslo_messaging.rpc.server [req-a4dbe1f9-420a-420f-9605-334d27573a10 5a05be334c0a4c88857ef9e0c2d39fee afe548f671f74763ad67715f68e5d7ac] Exception during message handling: TypeError: exceptions must be old-style classes or derived from BaseException, not NoneType
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 187, in wrapped
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server return f(self, *args, **kwargs)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/utils.py", line 571, in wrapper
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server return func(self, *args, **kwargs)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 3353, in update_access
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server share_server=share_server)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/access.py", line 283, in update_access_rules
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server share_server=share_server)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/access.py", line 322, in _update_access_rules
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server share_server)Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/access.py", line 390, in _update_rules_through_share_driver
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server share_server=share_server
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/cephfs/driver.py", line 257, in update_access
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server share_server=share_server)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/ganesha/__init__.py", line 282, in update_access
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server self.ganesha.add_export(share['name'], confdict)
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/drivers/ganesha/manager.py", line 484, in add_export
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server self._mkindex()
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server TypeError: exceptions must be old-style classes or derived from BaseException, not NoneType
Jul 03 18:01:45 devstack-host-vm.rdocloud manila-share[520947]: 2018-07-03 18:01:45.444 TRACE oslo_messaging.rpc.server

Tom Barron (tpb)
Changed in manila:
assignee: nobody → Tom Barron (tpb)
importance: Undecided → Low
milestone: none → rocky-3
tags: added: cephfs driver ganesha
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/579949

Changed in manila:
status: New → In Progress
Revision history for this message
Tom Barron (tpb) wrote :
Download full text (6.3 KiB)

With the patch from review 579949 we get a cleaner stack trace for the same error as above and the actual issue (unknown dbus service) is exposed:

Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: Command: sudo manila-rootwrap /etc/manila/rootwrap.conf dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.d/share-1a5da8d5-366f-4762-a9c1-e2da6bf5996e.conf.WTomPm string:EXPORT(Export_Id=1005)
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: Exit code: 1
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: Stdout: u''
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: Stderr: u'Error org.freedesktop.DBus.Error.ServiceUnknown: The name org.ganesha.nfsd was not provided by any .service files\n'
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 187, in wrapped
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server return f(self, *args, **kwargs)
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/utils.py", line 571, in wrapper
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server return func(self, *args, **kwargs)
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server File "/opt/stack/manila/manila/share/manager.py", line 3353, in update_access
Jul 03 18:33:51 devstack-host-vm.rdocloud manila-share[524687]: 2018-07-03 18:33:51.737 TRACE oslo_messaging.rpc.server share_server=share...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.openstack.org/579949
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=1f160196e40d129aca6713e9d2e014d540b4541c
Submitter: Zuul
Branch: master

commit 1f160196e40d129aca6713e9d2e014d540b4541c
Author: Tom Barron <email address hidden>
Date: Tue Jul 3 14:50:59 2018 -0400

    Fix bare exceptions in ganesha manager

    Closes-Bug: #1779922
    Change-Id: Ife18c61c4d195a73980a03287700df7ba9b52b9c

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/580848

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/queens)

Reviewed: https://review.openstack.org/580848
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=f9992b9059eff97b76409ccfd8a94a4d8cea9852
Submitter: Zuul
Branch: stable/queens

commit f9992b9059eff97b76409ccfd8a94a4d8cea9852
Author: Tom Barron <email address hidden>
Date: Tue Jul 3 14:50:59 2018 -0400

    Fix bare exceptions in ganesha manager

    Closes-Bug: #1779922
    Change-Id: Ife18c61c4d195a73980a03287700df7ba9b52b9c
    (cherry picked from commit 1f160196e40d129aca6713e9d2e014d540b4541c)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 6.0.2

This issue was fixed in the openstack/manila 6.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 7.0.0.0b3

This issue was fixed in the openstack/manila 7.0.0.0b3 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.