periodic-tripleo-ci-centos-7-standalone-full-tempest-master fails during tempest run ImagesTestJSON:test_delete_saving_image

[Marios Andreou]

master promotion blocker

During both openstack-periodic-master runs today at [1][2] the periodic-tripleo-ci-centos-7-standalone-full-tempest-master fails during the tempest.api.compute.images tests specifically test_delete_saving_image with trace like:

        Traceback (most recent call last):
          File "/usr/lib/python2.7/site-packages/tempest/api/compute/images/test_images.py", line 55, in test_delete_saving_image
            wait_until='SAVING')
          File "/usr/lib/python2.7/site-packages/tempest/api/compute/base.py", line 346, in create_image_from_server
            waiters.wait_for_image_status(client, image_id, wait_until)
          File "/usr/lib/python2.7/site-packages/tempest/common/waiters.py", line 181, in wait_for_image_status
            raise lib_exc.TimeoutException(message)
        tempest.lib.exceptions.TimeoutException: Request timed out
        Details: (ImagesTestJSON:test_delete_saving_image) Image a507b352-0c86-4b3e-b614-45cff4eaaa46 failed to reach SAVING state (current state ACTIVE) within the required time (300 s).

[1] http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/85662f6/logs/stestr_results.html
[2] http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/35fea03/logs/stestr_results.html

[Alan Bishop (alan-bishop)]

Looks like a repeat of https://bugs.launchpad.net/tempest/+bug/1713163; see comment 5 for suggestion on updating the tempest test's definition of success. Basically, don't require observing the transitional SAVING state.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/699624

[Marios Andreou (marios-b)]

got a green run of periodic-tripleo-ci-centos-7-standalone-full-tempest-master at https://review.rdoproject.org/r/#/c/24224/ and master promoted with it.

and indeed as per https://bugs.launchpad.net/tempest/+bug/1713163/comments/5 since it is timing based this is inconsistent/race condition.

I'm going to propose adding this to the skip list until the test can be improved?

there https://review.opendev.org/699624 Add tempest.api.compute.images test_delete_saving_image to skip

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/699624
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=50ae41cb841ab9909b15b35c4b246c47acde1e29
Submitter: Zuul
Branch: master

commit 50ae41cb841ab9909b15b35c4b246c47acde1e29
Author: Marios Andreou <email address hidden>
Date: Wed Dec 18 10:49:46 2019 +0200

    Add tempest.api.compute.images test_delete_saving_image to skip

    Per related bug this adds test_delete_saving_image to maste skip
    list as it often fails on race condition.

    Change-Id: Ifea780d81ead74a25131a8f3f179cb967081c265
    Related-Bug: 1856723

[Marios Andreou (marios-b)]

skip merged and looks more stable at https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-standalone-full-tempest-master

not sure we want to move this to fix-released since we just skipped the test. can keep the bug until test is unskipped

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/706817
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=13fffef03629b54913c8ed0654633f80746af514
Submitter: Zuul
Branch: master

commit 13fffef03629b54913c8ed0654633f80746af514
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Mon Feb 10 12:56:31 2020 +0000

    Revert "Add tempest.api.compute.images test_delete_saving_image to skip"

    Depends-On: https://review.opendev.org/#/c/702808/

    This reverts commit 50ae41cb841ab9909b15b35c4b246c47acde1e29.

    Closes-Bug: #1856723

    Change-Id: I56a5cde56422093d7a2ddd30699cfcf3b5afd85f