@Matthew:
I and izadorozhna, we fully understand your point - stick to the same mechanisms that already exist and support it.
We added skips for keystone admin tests against 'publicUrl' because we want to avoid false negative test results. I hope that it is obvious that keystone admin tests will fail against keystone 'publicUrl'. So we think that skipping these tests is more reasonable, than lots of failures.
Also, we added skips for keystone non-admin tests against 'adminUrl', because some tests for public Keystone API fail, if you point adminUrl in tempest config. For example:
test_tenants (list_tenants) FAIL: 403 error, log here : http://paste.openstack.org/show/380621/
reason: <admin-endpoint>/v2.0/tenants non reachable for non-admin user
test_users (change passwd) FAIL: 401, 404 for :35357/v2.0/OS-KSCRUD/users/<id>, log here: http://paste.openstack.org/show/380622/
reason: <admin-endpoint>/v2.0/OS-KSCRUD/users/<id> not exists on non-admin endpoint
Sure thing, we can skip only these two tests, but as for me, it is a bad idea to check that admin endpoint can act almost like a public endpoint.
It is clear, that our suggestion (skips) is not only one possible solution and definitely not perfect. But all other simple solutions look like dirty hacks(raw_request, one_more_identity_client, etc). In my humble opinion, the best solution here is allowing tests to choose requered endpoint, adminUrl for admin tests and publiclUrl for non-admin tests. Is definitely not the same mechanisms that already exist, because only keystone API requires such complicated logic. So, we decided to not propose such complicated solution.
@Matthew:
I and izadorozhna, we fully understand your point - stick to the same mechanisms that already exist and support it.
We added skips for keystone admin tests against 'publicUrl' because we want to avoid false negative test results. I hope that it is obvious that keystone admin tests will fail against keystone 'publicUrl'. So we think that skipping these tests is more reasonable, than lots of failures.
Also, we added skips for keystone non-admin tests against 'adminUrl', because some tests for public Keystone API fail, if you point adminUrl in tempest config. For example: paste.openstack .org/show/ 380621/ endpoint> /v2.0/tenants non reachable for non-admin user v2.0/OS- KSCRUD/ users/< id>, log here: http:// paste.openstack .org/show/ 380622/ endpoint> /v2.0/OS- KSCRUD/ users/< id> not exists on non-admin endpoint
test_tenants (list_tenants) FAIL: 403 error, log here : http://
reason: <admin-
test_users (change passwd) FAIL: 401, 404 for :35357/
reason: <admin-
Sure thing, we can skip only these two tests, but as for me, it is a bad idea to check that admin endpoint can act almost like a public endpoint.
It is clear, that our suggestion (skips) is not only one possible solution and definitely not perfect. But all other simple solutions look like dirty hacks(raw_request, one_more_ identity_ client, etc). In my humble opinion, the best solution here is allowing tests to choose requered endpoint, adminUrl for admin tests and publiclUrl for non-admin tests. Is definitely not the same mechanisms that already exist, because only keystone API requires such complicated logic. So, we decided to not propose such complicated solution.