keystone tests git clone keystoneclient

[Ionuț Arțăriși]

Running the unit tests in keystone using the run_tests.sh command uses python's subprocess to manually git clone keystoneclient from review.openstack.org.

Why is this done? Why isn't pip-requires being used instead? This makes it hard to run tests in a controlled environment.

We would like to test keystone against our own packaged keystoneclient. Also, is there any reason why review.openstack.org is used instead of github.com/openstack?

[Ionuț Arțăriși (mapleoin)]

Looking closer I saw that there are two different revisions of python-keystoneclient being tested here. Which answers my first question I guess. But I still think this should be done more cleanly.

Shouldn't the tests in 'tests/test_keystoneclient.py' be moved to the python-keystoneclient repository?

[Rafael Durán Castañeda (rafadurancastaneda)]

I think this duplicates https://bugs.launchpad.net/keystone/+bug/948495 and you might get your answer there.

[Ionuț Arțăriși (mapleoin)]

Thanks Rafael. That's a very good reference. I now understand why it was done like this, but I don't think that issue was solved. The fix on that bug is more like a hasty hack to just get over it.

Ideally we would still be able to run the functional tests against keystoneclient, but it would be against our installed version of keystoneclient, not the one in git. This is easily accomplished currently by just not running the git clone command which makes the tests import the system-installed keystoneclient.

The problem is that we would have to disable/delete one of these two test classes: KcMasterTestCase or KcEssex3TestCase in test_keystoneclient.py. Otherwise one of them would always fail because the corresponding keystoneclient version is not installed on the system.

Even from the name of those test classes it's clear that something is wrong. Those are basically versioned classes (KcMasterTestCase, KcEssex3TestCase). That versioning information is already present in keystoneclient's git.

So I guess this question remains: why aren't the individual keystoneclient versions tested against keystone rather than testing keystone against all the existing keystoneclient versions?