multi_backend tempest tests are failing

[Valeriy Ponomaryov]

Now all multi_backend CI jobs in gerrit are failing with trace:

ft1.88: setUpClass (tempest.api.share.admin.test_multi_backend.ShareMultiBackendTest)_StringException: Traceback (most recent call last):
  File "tempest/api/share/admin/test_multi_backend.py", line 53, in setUpClass
    cls.shares = cls.create_shares(share_data_list)
  File "tempest/api/share/base.py", line 379, in create_shares
    raise e
ShareBuildErrorException: Share 8f86355e-45fc-43db-90f1-c6a3e40b735a failed to build and is in ERROR status

But, manila-share does know nothing about such error:

http://logs.openstack.org/89/146089/4/check/gate-manila-tempest-dsvm-neutron-multibackend/189c044/logs/testr_results.html.gz

http://logs.openstack.org/89/146089/4/check/gate-manila-tempest-dsvm-neutron-multibackend/189c044/logs/screen-m-shr.txt.gz

It is discovered, that fails share creation for second backend for all "retries". Also, no errors within m-shr service.

In manila-scheduler we have following errors for all retries:
extra_spec requirement 'PARIS' does not match 'LONDON' _satisfies_extra_specs
...
Failed to schedule_create_share: No valid host was found. .

Looks like second backend is not starting anymore.

From manila.conf

enabled_share_backends = london

So, second backend was not enabled for some reason.

In localrc we have:
MANILA_MULTI_BACKEND=True

That should be OK.

But within devstack run it was not read somehow.

Root cause:
devstack commit
https://github.com/openstack-dev/devstack/commit/537532931da4103e1a873684476dac6d7fe56489
made changes, that lead to incompatibility with "trueorfalse" func.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/148253

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/148253
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=8747c6623e46c6c99756969ec5d017b393ad4a98
Submitter: Jenkins
Branch: master

commit 8747c6623e46c6c99756969ec5d017b393ad4a98
Author: vponomaryov <email address hidden>
Date: Mon Jan 19 15:06:38 2015 +0200

    Fix devstack compatibility

    Devstack commit 537532931da4103e1a873684476dac6d7fe56489 changed behaviour
    of devstack func 'trueorfalse' so, our devstack plugin started behave with
    error where this func was used.

    Change-Id: I646b66102f60e9a091e09f43882413a9941ec012
    Closes-Bug: #1412409