test_delete_saving_image fails because image hasn't transitioned to SAVING

[Ihar Hrachyshka]

http://logs.openstack.org/33/440933/4/check/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/ebfa453/logs/testr_results.html.gz

Traceback (most recent call last):
  File "tempest/api/compute/images/test_images.py", line 48, in test_delete_saving_image
    wait_until='SAVING')
  File "tempest/api/compute/base.py", line 301, in create_image_from_server
    image_id, wait_until)
  File "tempest/common/waiters.py", line 179, in wait_for_image_status
    raise lib_exc.TimeoutException(message)
tempest.lib.exceptions.TimeoutException: Request timed out
Details: (ImagesTestJSON:test_delete_saving_image) Image 93090479-409d-49f9-b2d3-2bc11a40f967 failed to reach SAVING state (current state ACTIVE) within the required time (196 s).

I briefly looked at the test case, and it seems to me that it may happen when the image snapshot is produced quicker than tempest being able to catch the intermittent transition. If so, the test case may need a refinement to check that a snapshot was produced and not that the state was in SAVING for some time.

[jichenjc (jichenjc)]

accordingly to avoid description, looks like this is a tempest issue?

[Ihar Hrachyshka (ihar-hrachyshka)]

As far as I understand, this test shouldn't even exist in scope of tempest, because you can't guarantee that backend will be slow to process the request. This kind of interaction would be better served by a test with better execution flow control. I am not sure if glance have those, but in neutron realm it would be a functional test with backend persistence artificially slowed down.

[Ihar Hrachyshka (ihar-hrachyshka)]

Added glance to the list of affected projects because I believe they will need to port the tempest test to somewhere else.

[Ihar Hrachyshka (ihar-hrachyshka)]

The failure happened lately here: http://logs.openstack.org/45/556545/1/check/legacy-tempest-dsvm-neutron-src/7b9f588/logs/testr_results.html.gz

[Brian Rosmaita (brian-rosmaita)]

Marking as invalid for Glance. The problem is occurring in a Compute API (nova) test. I think your analysis is correct, that this is a timing problem and that this may be too fine-grained a test; the test should use a different success criteria.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/702808

[Ghanshyam Mann (ghanshyammann)]

Yeah, Tempest should not assert on timing operation or at least should not assume the operartion timing as test critria.

This test is to check if image in SAVING state can be deleted or not. I will say modify the tests to run only if image is in SAVING state while deleting otherwise skip.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/702808
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=faa163442beca2844183f25443e3168a87dbeaa8
Submitter: Zuul
Branch: master

commit faa163442beca2844183f25443e3168a87dbeaa8
Author: Soniya Vyas <email address hidden>
Date: Wed Jan 15 19:34:57 2020 +0530

    Added fix to 'test_delete_saving_image' testcase

    'test_delete_saving_image' fails because image
    snapshot is produced quicker than tempest being able
    to catch the 'SAVING' transition.

    Hence, the testcase needs following refinement:-
    1. testcase should catch the state of image in 'SAVING'
       state only and then delete the image.
       By this case, we can get the idea that image is
       deleted in 'SAVING' state only and the testcase will
       achieve its motto.
    2. If the image trasits to a state other than
       'SAVING', testcase can just delete the image.

    Related-Bug: #1713163
    Signed-off by: Soniya Vyas<email address hidden>
    Change-Id: I50606c67e9f626935878aaa61d0adbc236272054

[Martin Kopec (mkopec)]

Marking as Fixed Committed as https://review.opendev.org/702808 got merged.

[Martin Kopec (mkopec)]

Fix Released as the change is part of tempest 24 tag already.