OpenStack Object Storage (swift)

test_ratelimit_sleep_with_incr unreliable in gate

Bug #1298154 reported by Clark Boylan on 2014-03-27

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Object Storage (swift)	Fix Released	Undecided	Unassigned	OpenStack Object Storage (swift) 1.13.1 "icehouse"

Bug Description

test_ratelimit_sleep_with_incr has failed in the gate. See http://logs.openstack.org/09/83209/3/gate/gate-swift-python26/5109caa/console.html for log of the failure. This appears to be related to the test using eventlet.sleep() and assuming the sleeps will no go over the desired time amount. But eventlet.sleep() can sleep for longer than specified.

To correct this time.time() and possibly eventlet.sleep() may need to be mocked out to allow for a greater degree of control over the "sleep" period.

While we are at it the assert can be rewritten as 'self.assertTrue(abs(500 - (time.time() - start) * 1000) < 100)' to use the same time scale as ratelimit_sleep.

Tags:

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-31: Fix merged to swift (master)

Reviewed: https://review.openstack.org/83277
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=5bb7603fce7bba58dd73cdca956e015054391380
Submitter: Jenkins
Branch: master

commit 5bb7603fce7bba58dd73cdca956e015054391380
Author: Peter Portante <email address hidden>
Date: Wed Mar 26 22:38:03 2014 -0400

Mock out time and sleep to avoid races

    We mock out time.time(), time.sleep() and eventlet.sleep() so that we
    avoid test problems caused by exceedingly long delays during the
    execution of the test.

We also make sure to convert the units used in the tests to
milliseconds for a bit more clarity.

Closes bug: 1298154

Change-Id: I803d06cbf205a02a4f7bb1e0c467d276632cd6a3

Changed in swift:
status:	New → Fix Committed

John Dickinson (notmyname) on 2014-04-03

Changed in swift:
milestone:	none → 1.13.1-rc1

Thierry Carrez (ttx) on 2014-04-04

Changed in swift:
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-04-08: Fix proposed to swift (feature/ec)

Fix proposed to branch: feature/ec
Review: https://review.openstack.org/85919

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-04-09: Fix merged to swift (feature/ec)

Download full text (13.1 KiB)

Reviewed: https://review.openstack.org/85919
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=58c0f5edbf2e61a422e00a4b5cdb3315d6ed7005
Submitter: Jenkins
Branch: feature/ec

commit 0e366aa92214234f77709640c3eecd8a24f108b4
Author: Peter Portante <email address hidden>
Date: Mon Mar 31 23:51:43 2014 -0400

Move module level setup under setup_package method

    Perform all the package level test setup under the setup_package
    method so that we only read the configuration file on disk when
    functional tests are run. This let's us "play nice" with other tests.

Change-Id: I699d664bcef6754aac4ec6877830e745b764342f

commit bf292ae0cb64c8af8c2e1c8d5cad6b37a59113b1
Author: gholt <email address hidden>
Date: Thu Apr 3 23:06:13 2014 +0000

Fixed error in container sync exc handling

    If container sync got a non-ClientException and then a ClientException
    during a get/put operation, it previously would break in the error
    handling.

Change-Id: Ib2fa70270f3ec870bf60d5bbb3e8f514aeeb0927

commit de020f018999e3a1decd59b73bc363ae21c701a3
Author: Peter Portante <email address hidden>
Date: Thu Apr 3 12:07:42 2014 -0400

Handle getting info on wrong database type

Change-Id: I32f66f6a7683180a18a2807143d0910c75bf16f0

commit eaf6815465159a793ad715c28dbfb5c0aae94c31
Author: John Dickinson <email address hidden>
Date: Wed Apr 2 16:58:40 2014 -0700

updates for 1.13.1 release

Change-Id: I9a84c476e8d370c3a5dbc24db77a7258d521013f

commit 39f5eab890394fce7e2efb9ffc1938d8156ed989
Author: Yuan Zhou <email address hidden>
Date: Tue Apr 1 15:21:19 2014 +0800

Clean up swift-{account, container}-info

Reuse common code; add unit tests; ensured coverage was at 100%.

Change-Id: Id6fcc7cb07fd178e00d43968e3e2cc03226fdc05

commit 44b39417baffb6047ca378e33d6694b5db7fc108
Author: Peter Portante <email address hidden>
Date: Mon Mar 31 23:22:49 2014 -0400

Read the configuration once for all func tests

    Merge the swift_testing module into the functional test module itself,
    so that we can read the configuration once for all unit tests, sharing
    the same constraints.

Change-Id: I9fbbfdade9adca329cd79f7d4291ba009327c842

commit 5ea6c88128ed966ba419848064680dec34850fda
Author: Peter Portante <email address hidden>
Date: Mon Mar 31 22:56:17 2014 -0400

Only change collation for functional tests

    The unconditional module-level change of the locale collation can
    affect other tests run under nose. To avoid this, make the change at
    the package level, and restore it when finished.

Change-Id: Iacda42aa7155f1cfa7d0dcadbf73b961dfb4b923

commit 07fcf50c3ac8d95d59901992affea82c45719340
Author: Peter Portante <email address hidden>
Date: Wed Apr 2 16:16:21 2014 -0400

Rework use of constraints to ease testing

    Prior to this patch both mainline code and testing modules imported
    and used constraints directly into their own namespace, or relied on
    the namespace of other modules that were not the constraints
    module. This meant that if a unit test wanted to change a constraint
...

Reviewed:  https://review.openstack.org/85919
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=58c0f5edbf2e61a422e00a4b5cdb3315d6ed7005
Submitter: Jenkins
Branch:    feature/ec

commit 0e366aa92214234f77709640c3eecd8a24f108b4
Author: Peter Portante <peter.portante@redhat.com>
Date:   Mon Mar 31 23:51:43 2014 -0400

Move module level setup under setup_package method
    
    Perform all the package level test setup under the setup_package
    method so that we only read the configuration file on disk when
    functional tests are run. This let's us "play nice" with other tests.
    
    Change-Id: I699d664bcef6754aac4ec6877830e745b764342f

commit bf292ae0cb64c8af8c2e1c8d5cad6b37a59113b1
Author: gholt <z-launchpad@brim.net>
Date:   Thu Apr 3 23:06:13 2014 +0000

Fixed error in container sync exc handling
    
    If container sync got a non-ClientException and then a ClientException
    during a get/put operation, it previously would break in the error
    handling.
    
    Change-Id: Ib2fa70270f3ec870bf60d5bbb3e8f514aeeb0927

commit de020f018999e3a1decd59b73bc363ae21c701a3
Author: Peter Portante <peter.portante@redhat.com>
Date:   Thu Apr 3 12:07:42 2014 -0400

Handle getting info on wrong database type
    
    Change-Id: I32f66f6a7683180a18a2807143d0910c75bf16f0

commit eaf6815465159a793ad715c28dbfb5c0aae94c31
Author: John Dickinson <me@not.mn>
Date:   Wed Apr 2 16:58:40 2014 -0700

updates for 1.13.1 release
    
    Change-Id: I9a84c476e8d370c3a5dbc24db77a7258d521013f

commit 39f5eab890394fce7e2efb9ffc1938d8156ed989
Author: Yuan Zhou <yuan.zhou@intel.com>
Date:   Tue Apr 1 15:21:19 2014 +0800

Clean up swift-{account, container}-info
    
    Reuse common code; add unit tests; ensured coverage was at 100%.
    
    Change-Id: Id6fcc7cb07fd178e00d43968e3e2cc03226fdc05

commit 44b39417baffb6047ca378e33d6694b5db7fc108
Author: Peter Portante <peter.portante@redhat.com>
Date:   Mon Mar 31 23:22:49 2014 -0400

Read the configuration once for all func tests
    
    Merge the swift_testing module into the functional test module itself,
    so that we can read the configuration once for all unit tests, sharing
    the same constraints.
    
    Change-Id: I9fbbfdade9adca329cd79f7d4291ba009327c842

commit 5ea6c88128ed966ba419848064680dec34850fda
Author: Peter Portante <peter.portante@redhat.com>
Date:   Mon Mar 31 22:56:17 2014 -0400

Only change collation for functional tests
    
    The unconditional module-level change of the locale collation can
    affect other tests run under nose. To avoid this, make the change at
    the package level, and restore it when finished.
    
    Change-Id: Iacda42aa7155f1cfa7d0dcadbf73b961dfb4b923

commit 07fcf50c3ac8d95d59901992affea82c45719340
Author: Peter Portante <peter.portante@redhat.com>
Date:   Wed Apr 2 16:16:21 2014 -0400

Rework use of constraints to ease testing
    
    Prior to this patch both mainline code and testing modules imported
    and used constraints directly into their own namespace, or relied on
    the namespace of other modules that were not the constraints
    module. This meant that if a unit test wanted to change a constraint
    for its operation, it had to know how that module was using the
    constraint, instead of referencing the constraint module itself.
    
    This patch unifies the use of constraints so that all constraints are
    referenced via the constraints module. In turn, this allows a test to
    leverage the re-loadable nature of the constraints in the constraints
    module.
    
    It addition, a number of functional tests where using the default
    values for constraints, instead of the configured value discovered in
    a test.conf or in an existing swift.conf. This patch removes those
    direct references in favor of the load_constraint() method from the
    test/functional/tests.py module.
    
    Change-Id: Ia5313d653c667dd9ca800786de59b59334c34eaa

commit 8a090916d11aff076fb919aa3782846579e53a8c
Author: Peter Portante <peter.portante@redhat.com>
Date:   Wed Apr 2 08:32:02 2014 -0400

Only fetch a timestamp when needed
    
    While it seems free to just call time.time() whenever it is
    convenient, in many cases code has to decide on a particular timestamp
    as a reference point for decisions. In other places, it just seems
    like a waste to make two calls that one would assume would return
    nearly identical times; however, certain test environments are
    notoriously overloaded (which is arguably a good thing, added stress
    to the code) where back-to-back calls to time.time() might produce
    timestamps seconds or even minutes apart.
    
    These simple changes here pick a point of reference for the operation
    to use throughout.
    
    Change-Id: Id2c0ffcc7f4ffafff61ebf59a8baeb079fd0c2b8

commit a7cfcc3d7a2b1973e8c5ba5b3782ab95a9a2a8fe
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Tue Apr 1 22:19:21 2014 -0600

Relocate DATADIR to backends
    
    It simply makes sense that the definition of DATADIR belongs to
    backends. After all, some of them may not even have any.
    
    Coincidentially, a few unnecessary imports are dropped.
    
    By the way, on the object server side, diskfile.py provides DATADIR
    in the same way already.
    
    Change-Id: I60bfd522c77c4a0ee13697a2e31141777c7e2398

commit 87710cc8a2654d872adb74d4fa4856c9beb41fbd
Author: Hou Ming Wang <houming.wang@easystack.cn>
Date:   Wed Feb 26 17:48:33 2014 +0800

AssertEquals is deprecated, use assertEqual partI
    
    assertEquals is deprecated in Python 2.7, need drop it.
    http://docs.python.org/2/library/unittest.html#deprecated-aliases
    
    Change-Id: Ieec505887f740b99b11ef40878ba3ee7247b78a8
    Closes-Bug: #1285040

commit 0b893825ebdd63d22d51cce42a3994f120f43921
Author: Chuck Thier <cthier@gmail.com>
Date:   Wed Mar 19 21:51:17 2014 +0000

Add "If-None-Match: *" support to PUT
    
    A common pattern that we see clients do is send a HEAD request before a
    PUT to see if it exists.  This can slow things down quite a bit
    especially since 404s on HEAD are currently a bit expensive.
    
    This change will allow a client to include a "If-None-Match: *" header
    with a PUT request.  In combination with "Expect: 100-Continue" this
    allows the server to return that it already has a copy of the object
    before any data is sent.
    
    I attempted to also include etag support with the If-None-Match header,
    but that turned up having too many hairy edge cases, so was left as a
    future excercise.
    
    DocImpact
    
    Change-Id: I94e3754923dbe5faba065719c7a9afa9969652dd

commit e6c910992d83611d579844d447a0cd47a6948b00
Author: David Goetz <dpgoetz@gmail.com>
Date:   Tue Apr 1 07:05:44 2014 -0700

Range requests not working with sub_SLOs
    
    Change-Id: I618c06e7f9251bfc7049c9dc6df66e539b90d9a9

commit 1e784a43327661a753dd420308671685f1460816
Author: Brian Cline <bcline@softlayer.com>
Date:   Mon Mar 31 13:12:57 2014 -0500

Fixes SUPER important typo in CHANGELOG.
    
    Change-Id: Id1ecc425cd9e32fef4d70e97d3a705a695eb73a2

commit c90ede29ff89ce3635e5553de4f3960a855cba5e
Author: Madhuri Kumari <madhuri.rai07@gmail.com>
Date:   Sun Mar 30 21:52:53 2014 +0530

Added swift-account-info tool.
    
    This is a very simple swift tool to retrieve information
    of an account that is located on the storage node.
    One can call the tool with a given account db file
    as it is stored on the storage node system.
    It will then return several information about that account.
    
    Change-Id: Ibfeee790adc000fc177b4b3c03d22ff785fda325

commit 5bb7603fce7bba58dd73cdca956e015054391380
Author: Peter Portante <peter.portante@redhat.com>
Date:   Wed Mar 26 22:38:03 2014 -0400

Mock out time and sleep to avoid races
    
    We mock out time.time(), time.sleep() and eventlet.sleep() so that we
    avoid test problems caused by exceedingly long delays during the
    execution of the test.
    
    We also make sure to convert the units used in the tests to
    milliseconds for a bit more clarity.
    
    Closes bug: 1298154
    
    Change-Id: I803d06cbf205a02a4f7bb1e0c467d276632cd6a3

commit b8f73a171d8d12c82a92435022d06755cd8a605b
Author: Peter Portante <peter.portante@redhat.com>
Date:   Wed Mar 26 22:10:29 2014 -0400

Move like unit tests together; add comments
    
    Some simple code movement to move the utils.ratelimit_sleep() unit
    tests together so that they can be viewed all at once.
    
    We also add some comments to document the behavior of
    utils.ratelimit_sleep(); small modification to max_rate parameter
    checking to match intended use.
    
    Change-Id: I3b11acfb6634d16a4b3594dba8dbc7a2d3ee8d1a

commit bb7bb4e03469ce3a79b5f3ecd1ef87e99530c5b0
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Mar 26 11:57:11 2014 -0700

Only run flake8 on Swift code
    
    In particular, this skips the "build" directory, which is prone to
    containing third-party stuff that doesn't comply with all the various
    hacking/flake8/pep8 checks.
    
    Change-Id: Ibe2240d8c6ca5947b2cac259cd7d0a1154f8a9cd

commit c6cebb6e621a245c9c2d5bff0df59689b0140373
Author: Florian Hines <syn@ronin.io>
Date:   Wed Mar 26 13:39:46 2014 -0700

Write out ring.gz's in a safer fashion
    
    Write the ring file to a temporary location first and then move it into
    place. This should help prevent issues like reading a partial ring file
    while a new instances is being written out.
    
    Change-Id: I28357a2f038a51cb9d823822b92f736dff033a9e

commit 31b994560332ffa22fbe45e43471db360467434c
Author: Peter Portante <peter.portante@redhat.com>
Date:   Tue Mar 25 15:56:44 2014 -0400

Invert which proxy logging middleware instance logs
    
    Prior to this patch the proxy_logging middleware always prepared to
    read the response and process it for access logging, where the
    proxy_logging instance first to handle the response bytes marked the
    request as "logged".
    
    This meant that the proxy_logging instance immediately before the
    proxy-server app (right most in the pipeline, if properly setup) would
    always log the responses for all the client requests, regardless if
    middleware to the left of it still had processing to do for the
    response. This would break SLO, where the slo middleware passed along
    the GET of the manifest, unchanged from the client, so that the right
    most proxy logging middleware would log the sponse for just the
    manifest, but then the slo middleware would fetch all the segments of
    the manifest, feeding them out in the response to the client, the
    request now marked as "logged" so that the left most proxy logging
    middleware would not log it.
    
    This patch inverts this behavior so that now the first proxy_logging
    middleware instance to receive a request in the pipeline marks that
    request as handling it. So now, the left most proxy_logging middleware
    handles logging for all client reaquests, and the right most
    proxy_logging middleware handles all other requests initiated from
    within the pipeline to its left.
    
    Closes bug: 1297438
    
    Change-Id: Ia3561523db76c693e4e0b2c38f461544dfcee086

commit 12c9dbe5f7090044c579e112e5616a9042d7f192
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed Mar 26 13:25:36 2014 -0700

minor cleanups for swift-container-info
    
    Change-Id: I4d0687f64524093e2ffaa4f8d31bcff70dca145e

commit 45e797b967bbb0c3d5023dd21e3576f1a9f872d0
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed Mar 12 13:00:21 2014 -0700

additional functional tests for account acls
    
    Change-Id: I7d8a8e48066281707862cd07f6b5a012476e8f24

commit 4b7bab9fc99e879a1e6342a21af1f14eb1eb1b8a
Author: Yuan Zhou <yuan.zhou@intel.com>
Date:   Wed Mar 26 10:23:53 2014 +0800

Update swift-get-nodes to use storage_directory util func
    
    Change-Id: I074d6c5f62e5218427c72b3e11de40da4ecfc0ab

commit a26b705d3991edbc770c73b47a34611ece0ae67d
Author: Greg Lange <greglange@gmail.com>
Date:   Tue Mar 18 21:17:52 2014 +0000

Fix TestGreenThreadSafeIterator test
    
    Change-Id: Ic121ae5c99b4e5c5e3d1efc432b55b762d50f3fb
    Closes-Bug: 1190244

commit cd149ae19f9235cf7bfc96ed2e06a4fd390a4006
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Mar 5 11:02:59 2014 -0800

Check object segment MD5s
    
    Now if a client is downloading a large object and one of the segment
    fetches gets bad data, we'll abort the request right away instead of
    letting the client carry on. We do this by checking each segment's
    body against the etag in the GET response's headers, like we tell
    clients to do.
    
    To make sure that the etag is the MD5, we add ?multipart-manifest=get
    to all segment GET requests. That way, a segment won't be another
    large object, because a large object's etag isn't its MD5. This does
    mean that clients can't make, say, a DLO manifest that points to a
    bunch of SLO manifests. However, that capability was only introduced
    in Swift 1.13.0 (post-Havana); prior to that, clients couldn't mix
    large-object types. I'm more than happy to trade that capability away
    in exchange for data-integrity checks.
    
    Change-Id: I98c78a291ee1a863a36d54a22867bd01c126644e

Thierry Carrez (ttx) on 2014-04-17

Changed in swift:
milestone:	1.13.1-rc1 → 1.13.1

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.