Functional tests fail: Expecting to find domain in project / auth_url was not provided to the Neutron client

[Bernard Cafarelli]

Functional tests started failing around end of September, some recent failures:
http://logs.openstack.org/57/509457/9/check/gate-neutronclient-test-dsvm-functional-ubuntu-xenial/63ed78d/
http://logs.openstack.org/41/507541/2/check/gate-neutronclient-test-dsvm-functional-ubuntu-xenial/afbcfbd/
And running tests on a local setup:
http://paste.openstack.org/show/622765/

Error message: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-xx)
For:
neutronclient.tests.functional.core.test_cli_formatter.TestCLIFormatter
neutronclient.tests.functional.core.test_common.CLICommonFeatureTest
neutronclient.tests.functional.core.test_purge.PurgeNeutronClientCLITest
neutronclient.tests.functional.core.test_readonly_neutron.SimpleReadOnlyNeutronClientTest
neutronclient.tests.functional.core.test_subnet_create.SubnetCreateNeutronClientCLITest

Error message: neutronclient.common.exceptions.NoAuthURLProvided: auth_url was not provided to the Neutron client
For:
neutronclient.tests.functional.core.test_clientlib.LibraryHTTPClientProjectTest
neutronclient.tests.functional.core.test_clientlib.LibraryHTTPClientTenantTest

[Akihiro Motoki (amotoki)]

Looks like the client used in the func test supports only keystone v2. We need to change the initialization way of the client class to use the way recommended by os-client-config which can handle keystone v2/v3 seemlessly.

[Akihiro Motoki (amotoki)]

Looks like this is a bug of tempest.lib.cli.base.CLIClient. It only accepts keystone v2 parameters and there seems no way to pass keystone v3 parameters [1]. Is there any suggestion?

What is the suggested way as neutron team? Should the neutron team implement our own way for the func test CLI initialization as OSC does?

[1] http://git.openstack.org/cgit/openstack/tempest/tree/tempest/lib/cli/base.py?id=f3eb9479e2e8ffd903b9ccb5ab8dff6b10b7194d#n98

[Luigi Toscano (ltoscano)]

Should we close this as duplicate of https://bugs.launchpad.net/tempest/+bug/1719687 and move the tags there? I'm going to do it for sahara-tests.

[Luigi Toscano (ltoscano)]

Or maybe we can keep this for the clients part.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/511419

[Andrea Frittoli (andrea-frittoli)]

Not a bug from my perspective, but something that would be good do implement soon.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/511419
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=2be24b43e990d0e7081ab38b8f937d11afa7d5a5
Submitter: Jenkins
Branch: master

commit 2be24b43e990d0e7081ab38b8f937d11afa7d5a5
Author: Akihiro Motoki <email address hidden>
Date: Thu Oct 12 08:09:29 2017 +0000

    Make func test work with keystone v3 only

    neutronclient func test is broken due to missing fully support
    of keystone v3 API. It seems keystone v2 API has been disbaled recently
    and this triggers the test failures.

    There are two causes.

    The one is because tempest.lib.cli.base supports only keystone v2 auth info.
    There is an on-going review in tempest, but until the next
    tempest release we need a workaround. The workaround is to pass
    keystone v3 related arguments in neutron() method.
    'flags' argument allows us to pass extra auth info to the CLI.

    The other reason is because HTTPClient (the legacy python binding class)
    supports only keystone v2 API. keystone v3 API is provided via SessionClient.
    I believe it is not a good idea to keep the two types of python binding
    classes in future. It looks like a time to drop HTTPClient.
    As a start, this commit drops HTTPClient func tests.
    Note that SessionClient supports keystone v2 as well.

    Closes-Bug: #1721553
    Related-Bug: #1719687
    Change-Id: I6f843e1412400bb1dfb4fc2352fc5cc4c3b837a4

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/519679

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-neutronclient 6.6.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519679
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=8cc59bf68883ffc9f52312a35d5eb75722a4deae
Submitter: Zuul
Branch: master

commit 8cc59bf68883ffc9f52312a35d5eb75722a4deae
Author: Akihiro Motoki <email address hidden>
Date: Tue Nov 14 15:31:56 2017 +0000

    Consume tempest CLIClient keystone v3 support

    tempest 17.1.0 added keystone v3 support to CLIClient and
    we no longer need a workaround for keystone v3 func tests.

    Change-Id: Ib32b5f82be938cd5213f6d71a0bf03cc1804587d
    Related-Bug: #1721553
    Related-Bug: #1719687

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/509487
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=d3db306af8b3c727c732fa3ac7475d43f911181e
Submitter: Zuul
Branch: master

commit d3db306af8b3c727c732fa3ac7475d43f911181e
Author: Luigi Toscano <email address hidden>
Date: Thu Jan 11 10:26:54 2018 +0100

    CLI tests: Identity v3 (project name, api version)

    Switch to --os-project-name instead of --os-tenant-name, and pass
    also --os-identity-api-version.
    All the clients which are wrapped by some method of the CLIClient class
    and which use cmd_with_auth support the newly introduced flags at least
    since Liberty.
    Only exception: the keystone client, which was removed in Newton, so
    it is not supported anyway (and it has not been fixed).

    Closes-Bug: #1721553
    Change-Id: I6d34e76b4089be024093a75f9e467d273d80f1c2

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/510631
Committed: https://git.openstack.org/cgit/openstack/sahara-tests/commit/?id=d1f9845bc76be752146531a5ee4f512fc7cc5e2e
Submitter: Zuul
Branch: master

commit d1f9845bc76be752146531a5ee4f512fc7cc5e2e
Author: Luigi Toscano <email address hidden>
Date: Thu Jan 11 15:37:48 2018 +0100

    CLI tests: proper support for Identity v3

    - pass user_domain_name and project_domain_name to CLIClient,
      which may be able to use it when the version of Tempest has
      I50490428f85e264e6b5e3b7c83ff768a986cb8c8;
    - use credentials.project_name if available, instead of the
      old credentials.tenant_name;
    - detect and pass the Identity API version, which may be used
      when I6d34e76b4089be024093a75f9e467d273d80f1c2 is merged.

    Closes-Bug: #1721553
    Closes-Bug: #1740853
    Depends-On: I6d34e76b4089be024093a75f9e467d273d80f1c2
    Change-Id: Idac157381be1fc503c13a10e69331227257ff649

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/sahara-tests 0.6.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tempest 18.0.0 release.