manila service will not start up if there are errors when allow access

[Peter Wang]

error was thrown while driver try to invoke driver's "allow_access":

2014-07-31 21:25:50.080 9138 ERROR manila.service [-] Unhandled exception
2014-07-31 21:25:50.080 9138 TRACE manila.service Traceback (most recent call last):
2014-07-31 21:25:50.080 9138 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 225, in _start_child
2014-07-31 21:25:50.080 9138 TRACE manila.service self._child_process(wrap.server)
2014-07-31 21:25:50.080 9138 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 202, in _child_process
2014-07-31 21:25:50.080 9138 TRACE manila.service launcher.run_server(server)
2014-07-31 21:25:50.080 9138 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 93, in run_server
2014-07-31 21:25:50.080 9138 TRACE manila.service server.start()
2014-07-31 21:25:50.080 9138 TRACE manila.service File "/opt/stack/manila/manila/service.py", line 374, in start
2014-07-31 21:25:50.080 9138 TRACE manila.service self.manager.init_host()
2014-07-31 21:25:50.080 9138 TRACE manila.service File "/opt/stack/manila/manila/share/manager.py", line 93, in init_host

as shown in ShareManager.py:

  for access_ref in rules:
                    if access_ref['state'] == access_ref.STATE_ACTIVE:
                        try:
                            self.driver.allow_access(ctxt, share,
                                                     access_ref,
                                                     share_server=share_server)
                        except exception.ShareAccessExists:
                            pass
            else:
                LOG.info(_("share %s: skipping export"), share['name'])

if a driver throw exception (not ShareAccessExists) in allow_access, then driver will refuse to start up

i am not sure if this is the expected behavior of share manager, but i think Manila should enhance the robust during startup
and even if a share down for any reason, we should be able to start up Manila Service so that admin has chance to remove/fix the problemic share in manila.

can we allow driver to throw exception for allow_access here so that share service could start up?

Thanks
Peter

[Julia Portnova (yportnova)]

Hi Peter,

Thanks for the bug report.
This is incorrect behavior of manila, service must start whether there are errors on exporting shares or not.
I think we need to add except Exception section, and log caughted error. And driver should not fail in this case.

[Peter Wang (peter.wang)]

Yes, i think so

Thanks
Peter

[Valeriy Ponomaryov (vponomaryov)]

Hi Peter,

What driver and share type were used? It will be useful for reproducing.

[Peter Wang (peter.wang)]

I am developing a new plugin for manila

in many cases, for example, manila driver could not re-allow access for the share due to internal error in nfs storage controller,
and throw a Exception other than exception.ShareAccessExists, then manila service would refuse to start up due to this unhanded exception.

Here is my opinion for exception handling:

self.driver.allow_access(ctxt, share,
                                                     access_ref,
                                                     share_server=share_server) shall be replaced by self.allow_access in manager.py
this enhancement will help in updating db access status if any error occured

how do you think?

Thanks
Peter

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/117814

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/117814
Committed: https://git.openstack.org/cgit/stackforge/manila/commit/?id=1a8f0860c35ae0f5a204bd18eddb6a93b1e96b78
Submitter: Jenkins
Branch: master

commit 1a8f0860c35ae0f5a204bd18eddb6a93b1e96b78
Author: Valeriy Ponomaryov <email address hidden>
Date: Fri Aug 29 18:17:21 2014 +0300

    Make m-shr more stable on start up

    When service m-shr is restarted it tries to ensure shares and its rules.
    If some error appears whole service fails to start up.
    Lets catch all exceptions, log them and let service start anyway.

    Closes-Bug: #1351195
    Change-Id: I979932b5a80ff2e1494c361887a1c1dea63daeb5