list images test has image-creation dependency

[Brian Rosmaita]

The following Images v2 API test:

tempest.api.image.v2.test_images.ListImagesTest.test_index_no_params

creates some images to use as part of the test. The test assumes that the Glance image upload capability is exposed to the user, but this may not be the case in a public cloud. This is a concern because the test is included among the DefCore tests. The key point is that this test should be focused on the ListImages functionality, not image creation.

[Rochelle Grober (rockyg)]

Test could be modified to check for existing glance images before performing the ListImages call. If images exist, then perform the list. If images do not exist, first create/load the images, then perform the list. This would allow the test to run against clouds that are already operational.

[Joe Hakim Rahme (rahmu)]

Agreed. This is not limited to test_index_no_params but to all the methods of the ListImagesTest.

[Joe Hakim Rahme (rahmu)]

Actually, looking into it, test_index_no_params could be checking for the existence of CONF.compute.image_ref.

The other methods listing based on parameters probably need to create their own images to make sure that the parameters will be find a match.

[Brian Rosmaita (brian-rosmaita)]

Joe and Rocky, thanks for looking at this.

I'm kind of worried that some of the tempest tests included in DefCore make assumptions about being run in a greenfield situation, or about the type of credentials being used during the test, or about how a cloud is configured, or ... . Rocky's suggestion will work to fix this test, but as Joe points out, there are going to be other images tests that will require setting up a test fixture so that you can make sure that the image properties you're looking for are there. I guess we'll have to cross that bridge when we come to it, since none of those tests are included in DefCore (yet).

[Matthew Treinish (treinish)]

So honestly this isn't going to change, we can't make a blind assertion about the environment the test is running in. You're ignoring that the tests have to run in isolation (and by default this means tempest creates a fresh tenant/project for each test class) and there is no guarantee that any tenant will have an images available for a list call. To ensure the list works properly in most environments. The image creation is part of the test setup for very good reasons, to construct a known state for use in later tests. (especially for the list filter tests which need sets of params) The test method itself is only testing the list operation given a known state created in the setup. This is not a unique pattern at all, almost every other list operation test is doing resource creation prior to doing the list.

Additionally, the tests inclusion in the defcore list isn't a justification for why this is a bug, it just explains why you are to looking at this. Just because your cloud doesn't support the only upstream mechanism that works for creating an image isn't a reason to say using it in a test as part of the environment setup is invalid. You would be unable to pass the rest of the image tests in tempest because the way your running your cloud which makes this a really weird thing to open a bug about.

[Brian Rosmaita (brian-rosmaita)]

Matthew,

Maybe this *is* kind of a weird thing to open a bug about, but the v2 ListImages test has a definite opinion about how to create an image. Even if you discount the Images v2 import task, a perfectly legitimate OpenStack technique for creating an image is the Compute API image-create action. So we've got several ways to create images:

- image create using Images v1 API
- image create using Images v2 API
- image import using Images v2 API (you can cross it out if you want, but it was working in J before the major refactoring in K)
- image-create action in Compute v2 API

There are no tests included in DefCore that address any of the above-listed capabilities, so DefCore doesn't have an opinion about a required OpenStack means of image-creation. So if the tempest test is OK as it stands, then it should not be included in DefCore, because it is imposing a requirement about a means of image-creation. But I guess you are right, that's a problem for DefCore, not for this particular test.

[Matthew Treinish (treinish)]

You opened a bug about tempest here, this is not a bug about that tests use in the defcore list. (I have separate opinions about that) When you just look at tempest we actually have tests for 3 of the 4 mechanisms you outlined, the only one which doesn't have tempest tests is the use of tasks. The list test in question uses v2 create because that is the only creation mechanism that works with glance v2 (it doesn't really matter that tasks worked in J) The v1 list test uses glance v1 for the create. You actually won't be able to use image import with tasks in tempest if it doesn't work in K (or now either) unless you backport all the necessary fixes for it to all the branches so it behaves consistently everywhere. (the gate will enforce this)

What you outlined about the multiple ways to create images in glance is actually total terribleness for end users and interop, especially with glance v2. How is an end user supposed to know which of the 4 mechanisms work on Cloud X, and what if they try to use the same code against Cloud Y which happens to only allow a different mechanism. It was honestly something I was hoping defcore was going to address by declaring one way to be the "OpenStack tm" way. (which I'm now starting to have doubts about) There really is not a tempest bug here, instead as just a proxy for the corporate political debate over the defcore list.