test_list_endpoints_for_token failed when heat is installed

[Xiaoming Kou]

I installed tempest and heat in one openstack environment. One case of Keystone is always failed
tempest.api.identity.admin.v2.test_tokens.TokensTestJSON.test_list_endpoints_for_token[id-ca3ea6f7-ed08-4a61-adbd-96906456ad31]

Now I can only work around this by uninstall heat before running Keystone tempest.

run: git clone -b 16.1.0 http://github.com/openstack/tempest.git
run: git clone -b ocata https://github.com/openstack/heat.git

run: ./tempest/tools/with_venv.sh pip --no-cache-dir install tempest
run: ./tempest/tools/with_venv.sh pip --no-cache-dir install -e heat

Traceback (most recent call last):
  File "/tmp/jenkins/workspace/end2end-master-compact-dvs-db/tempest/tempest/api/identity/admin/v2/test_tokens.py", line 133, in test_list_endpoints_for_token
    self.assertIn(service, service_names)
  File "/tmp/jenkins/workspace/end2end-master-compact-dvs-db/tempest/.venv/local/lib/python2.7/site-packages/testtools/testcase.py", line 417, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/tmp/jenkins/workspace/end2end-master-compact-dvs-db/tempest/.venv/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 'heat_plugin' not in [u'placement', u'neutron', u'cinderv2', u'cinderv3', u'glance', u'nova_legacy', u'heat-cfn', u'cinder', u'heat', u'nova', u'keystone']

[Andrea Frittoli (andrea-frittoli)]

The test should only look at Tempest services. We don't have a strong contract that the "service-available" setting must match a catalogue entry which is the reason for this failure.

[Ghanshyam Mann (ghanshyammann)]

that test is not doing the right things. it compare all services present in service_available config options which is not controlled by tempest only. tempest plugin can extend that options to skip/run their tests.

Test should check only service_available for tempest registered services only.

[Andrea Frittoli (andrea-frittoli)]

In general the service available flag name does not necessarily match the service name in the endpoint. That test only checks that Tempest is configured properly so I would propose to remove it completely.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/535270

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/535270
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=60cb4b2576ba86fa5b7f1cdf0bb8d372039c26f3
Submitter: Zuul
Branch: master

commit 60cb4b2576ba86fa5b7f1cdf0bb8d372039c26f3
Author: Andrea Frittoli (andreaf) <email address hidden>
Date: Thu Jan 18 10:11:35 2018 +0000

    Restrict test_list_endpoints_for_token to tempest

    The test_list_endpoints_for_token test now checks for services in
    service_available, however that may include services from plugins.
    We don't have a strong contract on the meaning of service_available
    so this is not correct as it will look in the catalog for things
    that may not even be OpenStack services.

    Fixing this by restricting by services in Tempest (including keystone).

    Change-Id: I32f4f056ea49fe04bfc309abe2cc9b9234b92847
    Closes-bug: #1743996

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tempest 18.0.0 release.