test_policy_IO_override from test.unit.proxy.test_server.TestObjectController randomly fails

[Mike Fedosin]

Sometimes I get this message from tox with "Finished: FAILURE" in the end.

15:12:46 ======================================================================
15:12:46 FAIL: test_policy_IO_override (test.unit.proxy.test_server.TestObjectController)
15:12:46 ----------------------------------------------------------------------
15:12:46 Traceback (most recent call last):
15:12:46 File "/home/jenkins/workspace/gate-swift-python27/test/unit/proxy/test_server.py", line 256, in wrapper
15:12:46 return f(*args, **kwargs)
15:12:46 File "/home/jenkins/workspace/gate-swift-python27/test/unit/proxy/test_server.py", line 1137, in test_policy_IO_override
15:12:46 self.assert_(now - 1 < float(e.timestamp) < now + 1)
15:12:46 AssertionError: False is not true

It always happens when my system is overloaded during the testing.

A quick look at the code showed that this test has a check:
        try:
            df.open()
        except DiskFileNotExist as e:
            now = time.time()
            self.assert_(now - 1 < float(e.timestamp) < now + 1)
        else:
            self.fail('did not raise DiskFileNotExist')

It seems that the code verifies that the exception's occurred and fails in the end if it's not. It's bad that the passage of this test is unpredictable and depends on the system load during the tests.

[Mike Fedosin (mfedosin)]

https://review.openstack.org/#/c/171593/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/171593
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=b76d8c5156c65e3ae6ead5213e40f70d80f7fa9d
Submitter: Jenkins
Branch: master

commit b76d8c5156c65e3ae6ead5213e40f70d80f7fa9d
Author: Mike Fedosin <email address hidden>
Date: Wed Apr 8 15:00:27 2015 +0300

    Fix broken test_policy_IO_override test

    Since the existing check depends on how quickly the test
    passes, it causes to the unpredictable behavior and may break
    the test.

    This commit removes the unnecessary dependency on the
    current time and leaves only the check that timestamp is
    greater than 0.

    Change-Id: I3321e9aba0df52cb78523710b26f310f35801674
    Closes-Bug: #1441599

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/175866

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/175866
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=5bb7c286ebb4a54e4d2bd5a02845644d1c651183
Submitter: Jenkins
Branch: feature/crypto

commit e440d6aed8a40848584767ed36811bf09c738838
Author: Kota Tsuyuzaki <email address hidden>
Date: Wed Apr 15 11:25:13 2015 -0700

    Fix best response to return correct status

    Current best response could return "503 Internal Server Error".
    However, "503" means "Service Unavailable". (The status int of
    Internal Server Error is 500)

    This patch fixes the response status as "503 Service Unavailable"

    Change-Id: I88b8c52c26b19e9e76ba3375f1e16ced555ed54c

commit 57011d5699d49a47ae89073ff27b39140ab4d652
Author: Ricardo Ferreira <email address hidden>
Date: Thu Mar 12 23:18:33 2015 +0000

    More user-friendly output for object metadata

    Split out system, user and other metadata in swift-object-info. Print
    every position line by line instead of raw dict representation, so it
    would be easier to parse with tools such as grep.

    Co-Authored-By: Ricardo Ferreira <email address hidden>
    Co-Authored-By: Kamil Rykowski <email address hidden>

    Change-Id: Ia78da518c18f7e26016700aee87efb534fbd2040
    Closes-Bug: #1428866

commit a162c2bdd7be12daa29dd07230f84efcaf1cab37
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Apr 16 06:06:35 2015 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I48ba06f4801ff2d7856d67e74d2b1f76c550fcf4

commit 52b102163e48dc04a6a492a3430efa1f7778d7ec
Author: Clay Gerrard <email address hidden>
Date: Wed Apr 15 15:31:06 2015 -0700

    Don't apply the wrong Etag validation to rebuilt fragments

    Because of the object-server's interaction with ssync sender's
    X-Backend-Replication-Headers when a object (or fragment archive) is
    pushed unmodified to another node it's ETag value is duped into the
    recieving ends metadata as Etag. This interacts poorly with the
    reconstructor's RebuildingECDiskFileStream which can not know ahead of
    time the ETag of the fragment archive being rebuilt.

    Don't send the Etag from the local source fragment archive being used as
    the basis for the rebuilt fragent archive's metadata along to ssync.

    Change-Id: Ie59ad93a67a7f439c9a84cd9cff31540f97f334a

commit 46bd6716ffae28aef53f15af170fd2df01b49843
Author: Kota Tsuyuzaki <email address hidden>
Date: Tue Apr 14 23:22:14 2015 -0700

    Small minor refactor on ec diskfile

    To be more helpful, add an inline comment and remove
    unnecessary assignment.

    Change-Id: Ia9c6993dfa03c238736955de8b0f5c1a7d5d1b65

commit 193fe9a5f906a2344bb5d328ad55b881e4086caa
Author: Lorcan <email address hidden>
Date: Wed Apr 15 11:32:32 2015 +0100

    Update swift-recon doc with more options

    The swit-recon tool has had several functional additions
    added recently but not all of these have been added to the docs.

    This change add...