Bug #1583756 “post-as-copy or COPY of SLO manifest results in wr...” : Bugs : OpenStack Object Storage (swift)

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2016-05-19:

#1

This is related to but different from https://bugs.launchpad.net/swift/+bug/1260446

Changed in swift:
assignee:	nobody → Alistair Coles (alistair-coles)

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2016-05-19:

#2

https://bugs.launchpad.net/swift/+bug/1260446 is actually fixed by copy middleware, commit 46d61a4

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-19: Fix proposed to swift (master)

#3

Fix proposed to branch: master
Review: https://review.openstack.org/318891

Changed in swift:
status:	New → In Progress

Alistair Coles (alistair-coles) on 2016-05-23

tags:

added: slo

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-25: Fix merged to swift (master)

#4

Reviewed: https://review.openstack.org/318891
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=d0ec1adb78b26f5c24312090796c69912e9e3da9
Submitter: Jenkins
Branch: master

commit d0ec1adb78b26f5c24312090796c69912e9e3da9
Author: Alistair Coles <email address hidden>
Date: Thu May 19 19:58:56 2016 +0100

Make SLO manifest copies retain correct content-type

    When copying an SLO manifest with multipart-manifest=get the actual
    manifest content-type should get copied to the destination, rather
    than the application/json value that is synthesised by SLO in a GET
    response. That way the result of a HEAD on the copied manifest is the
    same as a HEAD to the source, and the container listings for the two
    are consistent.

    This patch also un-skips a functional test and adds functional tests
    that verify this patch and also verify that etags and size also get
    correctly copied and updated in destination container (bug #1260446).

Closes-Bug: #1260446
Closes-Bug: #1583756

Change-Id: Ie7fa82f70b3ec3ef568f5355c69f6bce460ba25d

Changed in swift:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-31: Fix proposed to swift (feature/hummingbird)

#5

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/323599

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-31: Fix merged to swift (feature/hummingbird)

#6

Download full text (84.7 KiB)

Reviewed: https://review.openstack.org/323599
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0330478b70d0a699a0f9c21ef87c7e639d92564b
Submitter: Jenkins
Branch: feature/hummingbird

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <email address hidden>
Date: Tue May 31 16:25:50 2016 +0200

Fixed typo

Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <email address hidden>
Date: Wed May 25 11:21:25 2016 -0700

Clarify overload best practices in admin guide

Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <email address hidden>
Date: Wed May 25 09:53:31 2016 +0200

Add dispersion --verbose example to admin guide

Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <email address hidden>
Date: Wed May 25 14:35:54 2016 +1000

Add ring-builder dispersion command to admin guide

    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.

Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <email address hidden>
Date: Tue May 24 13:37:58 2016 -0700

Add links to mitaka install guides

Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <email address hidden>
Date: Fri Apr 29 12:12:00 2016 -0500

Allow concurrent bulk deletes

    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.

    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.

    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.

    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:

delete_concurrency = 1

This may be done prior to upgrading Swift.

    UpgradeImpact
    Closes-Bug: 1524454
    Change-Id: I128374d74a4cef7a479b221fd15eec785cc4694a

commit 226557afc42c245e050d84162497f46341407ef7
Author: Tim Burke <email address hidden>
Date: Thu May 19 18:55:40 2016 -0700

Turn on H703, so our translators don't punch us

Change-Id: I4ce3068f79563e4d4296c6e1078bc12f0cf84c96
Related-Bug: 1559431

commit 7b706926a8ed5bbcec3a678e868e301c9a6ed8f1
Author: Alistair Coles <email address hidden>
Date: Mon May ...

Reviewed:  https://review.openstack.org/323599
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0330478b70d0a699a0f9c21ef87c7e639d92564b
Submitter: Jenkins
Branch:    feature/hummingbird

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Tue May 31 16:25:50 2016 +0200

Fixed typo
    
    Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed May 25 11:21:25 2016 -0700

Clarify overload best practices in admin guide
    
    Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed May 25 09:53:31 2016 +0200

Add dispersion --verbose example to admin guide
    
    Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Wed May 25 14:35:54 2016 +1000

Add ring-builder dispersion command to admin guide
    
    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.
    
    Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue May 24 13:37:58 2016 -0700

Add links to mitaka install guides
    
    Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Apr 29 12:12:00 2016 -0500

Allow concurrent bulk deletes
    
    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.
    
    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.
    
    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.
    
    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:
    
       delete_concurrency = 1
    
    This may be done prior to upgrading Swift.
    
    UpgradeImpact
    Closes-Bug: 1524454
    Change-Id: I128374d74a4cef7a479b221fd15eec785cc4694a

commit 226557afc42c245e050d84162497f46341407ef7
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu May 19 18:55:40 2016 -0700

Turn on H703, so our translators don't punch us
    
    Change-Id: I4ce3068f79563e4d4296c6e1078bc12f0cf84c96
    Related-Bug: 1559431

commit 7b706926a8ed5bbcec3a678e868e301c9a6ed8f1
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Mon May 23 15:20:06 2016 +0100

Fix setup of manifest responses in SLO tests
    
    The swift_bytes param is removed from the content-type
    in the proxy object controller, so the SLO unit tests should
    not be registering GET responses with FakeSwift that have
    swift_bytes appended to the content-type.
    
    Nor should submanifest segment dicts have swift_bytes appended to
    their content-type values.
    
    Also adds a test for the object controller and container server
    handling of SLO swift_bytes.
    
    Change-Id: Icf9bd87eee25002c8d9728b16e60c8347060f320

commit d0ec1adb78b26f5c24312090796c69912e9e3da9
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu May 19 19:58:56 2016 +0100

Make SLO manifest copies retain correct content-type
    
    When copying an SLO manifest with multipart-manifest=get the actual
    manifest content-type should get copied to the destination, rather
    than the application/json value that is synthesised by SLO in a GET
    response. That way the result of a HEAD on the copied manifest is the
    same as a HEAD to the source, and the container listings for the two
    are consistent.
    
    This patch also un-skips a functional test and adds functional tests
    that verify this patch and also verify that etags and size also get
    correctly copied and updated in destination container (bug #1260446).
    
    Closes-Bug: #1260446
    Closes-Bug: #1583756
    
    Change-Id: Ie7fa82f70b3ec3ef568f5355c69f6bce460ba25d

commit 1b7001adeb935893017ff2428291d7edf9470b6b
Author: Brian Cline <bcline@us.ibm.com>
Date:   Sat May 21 03:54:33 2016 -0500

Improve linting for third-party middleware
    
    Currently any third-party CI doing linting of their own middlewares
    against the Swift codebase don't do so hot -- if they use swob's
    Response.status_int in any way, the linter will fail because it only
    gets set when status does.
    
    This patch defaults it to None in the class definition, seeing as
    a linter can't truly know that status_int will only be created lazily
    at runtime, and further, that its existence is tied to setting
    Response.status (via __init__ or setting the property directly).
    
    I couldn't actually find a code path within Swift where we could end
    up with a None when we access it; at init time, we default the status
    code to 200, and there don't seem to be any code paths that init with
    a None status code. (You might try, but you can't do that even today.)
    As a result, this seems to have no ill effect on anything, but is a
    huge help for linters.
    
    This doesn't seem to break any unit tests, func tests via v1, v2, and
    v3 auth, nor tempest tests via v2 or v3.
    
    Change-Id: Id28fb56bc7133b9eb59df7989a71b892fd919643

commit 14eb1803e931c5c501d99e612a6d59a57eccf464
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu May 12 23:57:49 2016 -0700

Fix probe failure and small things
    
    This is follow up for https://review.openstack.org/#/c/283351/.
    
    Probe fix:
    - The probe in the patch now fails (sometimes success luckily)
      because inbound X-Timestamp is deprecated at the change,
      f581fccf71034818d19062593eeb52a4347bb174, so we can not use
      X-Timestamp to make an object with arbitrary timestamp anymore
      from outside of Swift. This patch makes the probe to use internal
      client to put the objects to make the inconsistent situation.
    
    Small things:
    
    - Enable expirer split brain test even if we have just one policy.
    - FAIL rather than ERROR if the object was expired incorrectly
    - ObjectBrainSplitter now uses the policy set at instance variable in
      default instead of random choice of ENABLED_POLICIES.
    
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    
    Change-Id: I757dbb0f1906932ef5d508b48b4120f2794b3d07

commit 9ae3ee5a8b6aa169e1f4179cee748c721f480229
Author: Alfredo Moralejo <amoralej@redhat.com>
Date:   Thu May 19 15:56:15 2016 +0200

Fix locale directory in MANIFEST.in
    
    Actual location is swift/locale, not locale.
    
    This makes build to not include .mo files if they have been created
    previously with compile_catalog.
    
    Change-Id: I907e9bd6dde57cb9ed718656e56a313e82bad128
    Closes-Bug: 1583618

commit 876df35f847e607901e2ed8a344c71c607495705
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Tue May 17 10:48:25 2016 +1000

disable_fallocate also disables fallocate_reserve
    
    Currently when disable_fallocate is true it disables calling the
    fallocate syscall, but it doesn't disable fallocate_reserve. This
    patch fixes this.
    
    This problem has caused functional tests to fail in our SAIOs, since
    SAIOs have disable_fallocate set but the fallocate_reserve space free
    checking was still being run creating 507 responses. This is thanks
    to the change in fallocate_reserve default changing from 0 to 1%.
    
    Because fallocate_reserve and disable_fallocate causes SAIO functional
    tests to fail a section called 'Known Issues' has been added to the
    SAIO developer documentation which includes a warning about
    using fallocate_reserve on SAIOs.
    
    Change-Id: I727bfb0861ea26fe2f16ad55f4d36ae088864d8f

commit 2744492f30c758a4f076e2b30eaaf1e2e7fa586c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Apr 27 13:31:11 2016 -0500

Use the same key for memcache and env['swift.infocache']
    
    When we were caching directly to the WSGI environment, it made sense to
    have different keys for the different caches. Now that we have a
    separate data structure for the per-request cache, however, we ought to
    be consistent.
    
    Change-Id: I199cba6e5fc9ab4205bba369e6a2f34fc5ce22d4

commit 4248123169f88bf62dbc5413ae2c15d53010482e
Author: venkatamahesh <venkatamaheshkotha@gmail.com>
Date:   Sun May 15 21:27:46 2016 +0530

Remove the invalid project link
    
    swiftsync project is no longer maintained and so
    is removed from the list
    
    Change-Id: Ieae67c728da16e5516babd33d2e5937b14738d6a
    Closes-Bug: #1581975

commit 1adc6047f0dc56caffb3ab552de948c720b50682
Author: Brian Ober <bober@us.ibm.com>
Date:   Fri May 13 16:43:50 2016 -0500

Bypass Account Mgmt Functional Cases When Disabled
    
    The testPUT case is failing when keystone was enabled and
    allow_account_management is set to True.   There were a few issues
    needing addressed.
    
    First the case was renamed to call out what it was actually doing which
    is verifying an error scenario for which a PUT on a storage account was
    not allowed.
    
    Second the case was running even when allow_account_management is
    enabled, which is incorrect.   It "accidently" works with TempAuth
    because it requires a reseller permission, so the Keystone failure here
    has more to do with not requiring a reseller permission to do a PUT on a
    storage account for which a user has an operator role on.
    
    The common sense fix here is to not execute this test case when
    allow_account_management is enabled.
    
    Change-Id: Id29f5ca48f92cd139535be7064107b8a61b02856

commit 1c88d2cb818cec07d12ac17be5820cdd769aea5d
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Feb 11 15:51:45 2016 -0800

Fix up get_account_info and get_container_info
    
    get_account_info used to work like this:
    
      * make an account HEAD request
    
      * ignore the response
    
      * get the account info by digging around in the request environment,
        where it had been deposited by elves or something
    
    Not actually elves, but the proxy's GETorHEAD_base method would take
    the HEAD response and cache it in the response environment, which was
    the same object as the request environment, thus enabling
    get_account_info to find it.
    
    This was extraordinarily brittle. If a WSGI middleware were to
    shallow-copy the request environment, then any middlewares to its left
    could not use get_account_info, as the left middleware's request
    environment would no longer be identical to the response environment
    down in GETorHEAD_base.
    
    Now, get_account_info works like this:
    
      * make an account HEAD request.
    
      * if the account info is in the request environment, return it. This
        is an optimization to avoid a double-set in memcached.
    
      * else, compute the account info from the response headers, store it
        in caches, and return it.
    
    This is much easier to think about; get_account_info can get and cache
    account info all on its own; the cache check and cache set are right
    next to each other.
    
    All the above is true for get_container_info as well.
    
    get_info() is still around, but it's just a shim. It was trying to
    unify get_account_info and get_container_info to exploit the
    commonalities, but the number of times that "if container:" showed up
    in get_info and its helpers really indicated that something was
    wrong. I'd rather have two functions with some duplication than one
    function with no duplication but a bunch of "if container:" branches.
    
    Other things of note:
    
      * a HEAD request to a deleted account returns 410, but
        get_account_info would return 404 since the 410 came from the
        account controller *after* GETorHEAD_base ran. Now
        get_account_info returns 410 as well.
    
      * cache validity period (recheck_account_existence and
        recheck_container_existence) is now communicated to
        get_account_info via an X-Backend header. This way,
        get_account_info doesn't need a reference to the
        swift.proxy.server.Application object.
    
      * both logged swift_source values are now correct for
        get_container_info calls; before, on a cold cache,
        get_container_info would call get_account_info but not pass along
        swift_source, resulting in get_account_info logging "GET_INFO" as
        the source. Amusingly, there was a unit test asserting this bogus
        behavior.
    
      * callers that modify the return value of get_account_info or of
        get_container_info don't modify what's stored in swift.infocache.
    
      * get_account_info on an account that *can* be autocreated but has
        not been will return a 200, same as a HEAD request. The old
        behavior was a 404 from get_account_info but a 200 from
        HEAD. Callers can tell the difference by looking at
        info['account_really_exists'] if they need to know the difference
        (there is one call site that needs to know, in container
        PUT). Note: this is for all accounts when the proxy's
        "account_autocreate" setting is on.
    
    Change-Id: I5167714025ec7237f7e6dd4759c2c6eb959b3fca

commit ce90a1e79e16472ff75d205c8d907a5991888b3a
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Jan 21 13:19:30 2016 -0800

Make info caching work across subrequests
    
    Previously, if you called get_account_info, get_container_info, or
    get_object_info, then the results of that call would be cached in the
    WSGI environment as top-level keys. This is okay, except that if you,
    in middleware, copy the WSGI environment and then make a subrequest
    using the copy, information retrieved in the subrequest is cached
    only in the copy and not in the original. This can mean lots of extra
    trips to memcache for, say, SLO validation where the segments are in
    another container; the object HEAD ends up getting container info for
    the segment container, but then the next object HEAD gets it again.
    
    This commit moves the cache for get_*_info into a dictionary at
    environ['swift.infocache']; this way, you can shallow-copy the request
    environment and still get the benefits from the cache.
    
    Change-Id: I3481b38b41c33cd1e39e19baab56193c5f9bf6ac

commit 30e39cc9fa6353be37cd3b5a515f09b42ceb3b20
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed May 4 12:02:07 2016 -0700

Skip SLO-reconciling probe test when SLO is off
    
    The probe test in question tries to make a manifest referencing
    unavailable objects and expects that to fail with a 400. If the SLO
    middleware is enabled, then it checks the segments, can't find some,
    and returns the 400. If it's disabled, the PUT succeeds and makes an
    object whose contents are some JSON blob. In the latter case, the
    probe test would fail because it expected a real SLO manifest but
    didn't find one.
    
    Now we skip the remainder of the test when we detect that SLO is not
    enabled.
    
    Change-Id: I3e7e8e98107608e675efc24156e703bc167458bb

commit 721c788b9a0ceb22ffac0b2c797890898096a9f4
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu May 12 18:57:31 2016 -0700

Import BrainSpliter directly in expirer probe
    
    Change-Id: Ib3cfe38d1b17c09c7248b5cbcfe4e2eadff7acd9

commit eda1b330f3cf7bf9dd643854d4a221b6f024884d
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Thu May 12 18:58:15 2016 +0000

fix docs word usage for large obj copy
    
    Change-Id: I2b6ca3f9277b344ff5b05a85304084a7edcbcb66

commit 6834547f668a4649f5a42b8a4edc43b844fd6bbe
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Fri Apr 22 14:55:35 2016 -0700

Clean up fallocate tests a little
    
    Change-Id: I01f1ad8ef0f8910718fd2fb30c9e8285358baf84

commit b430c384db38ea1153f52d0accf0561dac1378a6
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu May 12 14:57:56 2016 +0100

Put back missing sysmeta unit tests
    
    We lost some unit tests from test_sysmeta.py in [1].
    This patch restores them. Since the proxy no longer
    implements post-as-copy or COPY then we need to plumb
    in a copy middleware to the test app in test_sysmeta.
    
    It is a little odd perhaps to have these tests under
    test/unit/proxy but the alternative is to duplicate
    the test setup and base test code that already exists
    in test_sysmeta.py into test_copy.py.
    
    [1] commit 46d61a4dcd9a5d9157625c06d6fe7d916e80c3d2
        (server side copy middleware)
    
    Change-Id: Iec02387ccbddbe3841a417880389c707cd5c0346

commit ce022f4417e973215b563412b504a8a1b9ce240c
Author: Saverio Proto <saverio.proto@switch.ch>
Date:   Thu May 12 15:30:17 2016 +0200

README: fix broken link
    
    There is a broken link to CONTRIBUTING.rst
    introduced in commit a829bd59770681f9d6c1ef02a6e1d5e441587a23
    
    Change-Id: Iaedf5ff3995229cf292202793809080f9f2c7fed

commit 9729bc83eb6845e65f8244b4104b2c1678f8fe38
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Apr 11 11:56:07 2016 +0200

Don't delete misplaced dbs if not replicated
    
    If one uses only a single replica and a database file is placed on a
    wrong partition, it will be removed instead of replicated to the correct
    partition.
    
    There are two reasons for this:
    1. The list of nodes is empty when there is only a single replica
    2. all(responses) is True even if there is no response at all, and the
    latter is always True if there is no node to replicate to.
    
    This patch fixes this by adding a special case if used with only one
    replica to the node selection loop and ensures that the list of
    responses is not empty.  Also adds a test that fails on current master
    and passes with this change.
    
    Closes-Bug: 1568591
    
    Change-Id: I028ea8c1928e8c9a401db31fb266ff82606f8371

commit 92274d479010f691004b1885e354224dc9c3b61c
Author: Andreas Jaeger <aj@suse.com>
Date:   Mon Mar 28 18:21:22 2016 +0200

List system dependencies for running common tests
    
    Add an other-requirements.txt file containing a cross-platform list of
    dependencies needed for running included tox-based tests. Also include a
    tox environment for convenience calling the bindep[*] utility to list
    any missing system requirements.
    
    Document bindep and other-requirements usage.
    
    This change is self-testing.
    
    For other-requirements.txt see also
    http://docs.openstack.org/infra/manual/drivers.html#package-requirements
    
    [*] http://docs.openstack.org/infra/bindep/
    
    Change-Id: Iea6f5fecba3b7cb9f6dac7029c0f17fc31fc0e3c

commit ad16e2c77bb61bdf51a7d3b2c258daf69bfc74da
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed May 11 19:54:47 2016 -0700

Stop complaining about auditor_status files
    
    Following fd86d5a, the object-auditor would leave status files so it
    could resume where it left off if restarted. However, this would also
    cause the object-reconstructor to print warnings like:
    
      Unexpected entity in data dir: u'/srv/node4/sdb8/objects/auditor_status_ZBF.json'
    
    ...which isn't actually terribly useful or actionable. The auditor will
    clean it up (eventually); the operator doesn't have to do anything.
    
    Now, the reconstructor will specifically ignore those status files.
    
    Change-Id: I2f3d0bd2f1e242db6eb263c7755f1363d1430048

commit 46d61a4dcd9a5d9157625c06d6fe7d916e80c3d2
Author: Prashanth Pai <ppai@redhat.com>
Date:   Wed Feb 18 11:59:31 2015 +0530

Refactor server side copy as middleware
    
    Rewrite server side copy and 'object post as copy' feature as middleware to
    simplify the PUT method in the object controller code. COPY is no longer
    a verb implemented as public method in Proxy application.
    
    The server side copy middleware is inserted to the left of dlo, slo and
    versioned_writes middlewares in the proxy server pipeline. As a result,
    dlo and slo copy_hooks are no longer required. SLO manifests are now
    validated when copied so when copying a manifest to another account the
    referenced segments must be readable in that account for the manifest
    copy to succeed (previously this validation was not made, meaning the
    manifest was copied but could be unusable if the segments were not
    readable).
    
    With this change, there should be no change in functionality or existing
    behavior. This is asserted with (almost) no changes required to existing
    functional tests.
    
    Some notes (for operators):
    * Middleware required to be auto-inserted before slo and dlo and
      versioned_writes
    * Turning off server side copy is not configurable.
    * object_post_as_copy is no longer a configurable option of proxy server
      but of this middleware. However, for smooth upgrade, config option set
      in proxy server app is also read.
    
    DocImpact: Introducing server side copy as middleware
    
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    Co-Authored-By: Thiago da Silva <thiago@redhat.com>
    
    Change-Id: Ic96a92e938589a2f6add35a40741fd062f1c29eb
    Signed-off-by: Prashanth Pai <ppai@redhat.com>
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit cb8cda6588735411c542a1e4d78315ee047a357b
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Mon May 9 20:28:42 2016 +0000

resurrect gholt blog posts on building consistent hashing ring
    
    These are blog posts authored by Greg Holt (gholt) and used with
    permission to add here. Content was only reformatted as rst and
    wrap prose lines at 70 characters.
    
    Change-Id: I7aa47c24b5019aa598ee005e01612a49514da25f

commit ba1a568f815a8a40bf75c0beb96b789bd07adcb6
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 9 10:48:54 2016 +0000

Rename hash_cleanup_listdir tests
    
    hash_cleanup_listdir was removed in [1], this patch
    renames all references to it in test_diskfile to refer to
    the cleanup_ondisk_files method that is now tested directly.
    
    Also remove the final references to the now non-existent
    function in a few comments.
    
    [1] I0b96dfde32b4c666eebda6e88228516dd693ef92
    
    Change-Id: I1e151799fc2774de9a1af092afff875af24a630c
    Related-Bug: #1550569

commit 6827affe62771949a041d958db17ba6cd29232ab
Author: John Dickinson <me@not.mn>
Date:   Tue Apr 5 11:35:22 2016 -0700

Rework the contributor docs
    
    This started as a new "new_contributor" doc. But we've already got
    at least 3 different docs like that.
    
    Change-Id: Ia2303ab55eeea01cc71acbccaeab55dad0ef5ff9

commit deaef2f9d62bb8ae7f4e8538e9fd9ddffbb8ebb2
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed May 4 16:48:28 2016 -0700

Remove unneeded setting of SO_REUSEADDR.
    
    This has been in eventlet.listen() since version 0.15.
    
    Change-Id: Ib77b28231a2180f1ea082f356c4687c39681a6f7

commit b3dd6a5df1ac4a5265b330787abcd52c7e116208
Author: Christopher Bartz <bartz@dkrz.de>
Date:   Tue May 3 14:33:05 2016 +0200

Improves log message in swift.obj.replicator
    
    Before this commit, when a local device has not found been found
    in a object-replication run, the policy was not mentioned in the
    error log. But it is of interest to know the policy, for example for
    error searching, when no local device has been found.
    
    Change-Id: Icb9f9f1d4aec5c4a70dd8abdf5483d4816720418

commit 99305b93000f28530731c05b34c69e0c7ce0463f
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon May 2 17:29:32 2016 -0700

Fix probe tests from commit cf48e75
    
    Commit cf48e75 changed the default account/container/object ports in a
    lot of places, including the probetests. However, it didn't change
    them in doc/saio/bin/remakerings, and since the probe tests must match
    the rings, they started failing.
    
    This commit just backs out the changes to the test/probe directory so
    that remakerings and the probe tests match again.
    
    Change-Id: I316a09e6ee1a911f37ce9df3d641644739f88eeb

commit cf48e75c259350daa667155e3ffd38b17946bf44
Author: Shashirekha Gundur <shashirekha.j.gundur@intel.com>
Date:   Mon Feb 1 18:06:54 2016 +0000

change default ports for servers
    
    Changing the recommended ports for Swift services
    from ports 6000-6002 to unused ports 6200-6202;
    so they do not conflict with X-Windows or other services.
    
    Updated SAIO docs.
    
    DocImpact
    Closes-Bug: #1521339
    Change-Id: Ie1c778b159792c8e259e2a54cb86051686ac9d18

commit 4c11833a9cbff499725365e535e217f3eae3c442
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Apr 28 11:53:51 2016 -0500

Remove ThreadPool class
    
    With the removement of threads_per_disk there is no longer a need to use
    run_in_thread() at all; it was just calling the function itself when
    running with 0 threads.
    Similar to force_run_in_thread() - with 0 threads it was basically doing
    the same like in tpool_reraise(), therefore replacing the call and
    finally removing the complete ThreadPool class.
    
    Note that this might break external consumers that are inheriting
    BaseDiskFileManager; in this case you need to adopt this change in your
    codebase then.
    
    Change-Id: I39489dd660935bdbfbc26b92af86814369369fb5

commit 33fdd0a356d83f12f63b9a94ff423ca815c96066
Author: Bryan Keller <kellerbr@us.ibm.com>
Date:   Fri Apr 22 10:03:23 2016 -0700

SwiftLogFormatter will log transaction IDs on INFO level
    
    Previously SwiftLogFormatter would make two checks. One to see if the
    transaction id was already in the message field and another check to
    make sure the log level wasn't set to info. If either of these was
    true, then it would not log the transaction ID in the transaction ID
    field.
    
    This commit removes the check for the info log. Now transaction IDs
    will be recorded in all cases that have them.
    
    Change-Id: Ic06538ab55a75d298169ae1745671573ee9c09e8
    Closes-Bug: #1504344

commit c6ac69c552f87d07cd98b6e3f54258913bdc90db
Author: ZhiQiang Fan <aji.zqfan@gmail.com>
Date:   Fri Apr 29 20:39:02 2016 +0800

[Trivial] Remove unnecessary executable privilege
    
    swift/common/storage_policy.py is not required to be executable.
    
    Change-Id: I733d70a88be25b32d8caf590dee4c53d14757fef

commit 96c1838a15645639154f1048285d53a891809937
Author: Brian Cline <bcline@us.ibm.com>
Date:   Fri Apr 29 05:00:36 2016 -0500

Update mailmap, resolve identity crisis
    
    Change-Id: Ia8e30a8c0f81068c259d5d08adad4f1a98460561

commit a67d91987dff4859924aeaa3ca8fe30190b4c210
Author: Brian Cline <bcline@us.ibm.com>
Date:   Fri Apr 29 04:01:39 2016 -0500

Fix minor typos in review guidelines
    
    Change-Id: I3acb3daeacb784c4038bf54d4103fab66c52f41a

commit 9d6a055b31f267e2380d0470021cac44cdf36203
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Apr 28 12:06:24 2016 -0500

Remove threads_per_disk setting
    
    This patch removes the threads_per_disk setting. It was already a deprecated
    setting and by default set to 0, which effectively meant to not use a per-disk
    thread pool at all. Users are encouraged to use servers_per_port instead.
    
    DocImpact
    
    Change-Id: Ie76be5c8a74d60a1330627caace19e06d1b9383c

commit a1ed76f1cbb3dcd5b4253920278b8bc4f32978a2
Author: ZhiQiang Fan <aji.zqfan@gmail.com>
Date:   Thu Apr 28 06:29:41 2016 +0800

Fix doc build if git is absent
    
    When building packages if git is absent, then we should not set
    html_last_updated_fmt. It can still be set via the -D switch
    when building with sphinx-build.
    
    Change-Id: I5d0b6cc87f27a052d6d0265546c0d347f00c4bb8
    Closes-Bug: #1552251

commit 00dd89fe6936e4c2da667b496a9996638c26196b
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Mar 3 06:08:12 2016 +0000

adding review guidelines
    
    Change-Id: I61304856a4ecccbbf3aa06c30822494592a3b3d5

commit 29544a9e175b1ec9e0dbfd5288edc00a1402d5ca
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Apr 27 16:59:00 2016 -0500

Use smaller quorum size in proxy for even numbers of replicas
    
    Requiring 2/2 backends for PUT requests means that the cluster can't
    tolerate a single failure. Likewise, if you have 4 replicas in 2
    regions, requiring 3/4 on a POST request means you cannot POST with
    your inter-region link down or congested.
    
    This changes the (replication) quorum size in the proxy to be at least
    half the nodes instead of a majority of the nodes.
    
    Daemons that were looking for a majority remain unchanged. The
    container reconciler, replicator, and updater still require majorities
    so their functioning is unchanged.
    
    Odd numbers of replicas are unaffected by this commit.
    
    Change-Id: I3b07ff0222aba6293ad7d60afe1747acafbe6ce4

commit 5e420efc728b9f02a3f15eb312ebfa29e2e221ff
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jan 7 15:46:17 2016 -0800

Fix versioned_writes functional test skipping
    
    Previously, if object versioning was enabled via the old-style
    allow_versions container-server setting rather than the new-style
    allow_versioned_writes proxy-server setting,
    TestCrossPolicyObjectVersioning would skip tests while
    TestObjectVersioning and TestObjectVersioningUTF8 would run them.
    
    Additionally, if versioned_writes was explicitly included in the
    proxy-server's pipeline and allow_versioned_writes was disabled,
    the functional tests would fail with a 412.
    
    Now, all three will use the same logic to check whether versioning is
    enabled. Specifically, they will all try to set an X-Versions-Location
    header and skip if it doesn't stick.
    
    Additionally, the TestCrossPolicyObjectVersioningEnv will now properly
    clean up after itself.
    
    Change-Id: I4c788a0e18587ff17d3c6e346fd22b881495f06d

commit 1a18079a15c2630e602c7c156a30e10e5f884d4b
Author: Denis V. Meltsaykin <dmeltsaykin@mirantis.com>
Date:   Thu Apr 7 23:07:13 2016 +0300

Change pids' type to set
    
    Previously, the pids type was list(), which raised a ValueError in case if
    pid was not in pids during the pids.remove(pid) call. The fix changes
    the type of pids to set() and substitutes remove() with discard(), which
    doesn't raise an Exception if a value is not in the set.
    
    Change-Id: I8d41cb2a8ec14d29e5c7411ddfe48ae7a41deb45
    Closes-Bug: #1567638

commit 0da9da513101c172d8923469e40f9c15b3518e73
Author: Andy McCrae <andy.mccrae@gmail.com>
Date:   Thu Mar 3 11:14:39 2016 +0000

Allow fallocate_reserve to be a percentage
    
    Add the ability to set the fallocate_reserve value as a percentage.
    This happens automatically when adding the '%' at the end of the value.
    Having the ability to set a % of free space rather than a byte value is
    useful especially when drive sizes are heterogenous.
    
    The default for fallocate_reserve has been adjusted to 1%, having the
    fallocate_reserve set seems sensible for all deploys and percentages are
    far safer to default than byte values (across drives of any size).
    
    Tests added for using fallocate_reserve as a percentage.
    
    Duplicate tests for fallocate_reserve have been removed.
    
    Docs updated to reflect the fallocate_reserve change.
    
    Change-Id: I4aea613a708205c917e81d6b2861396655e73238

commit 7d7eaab5afa4c36f0ac8467784138fc423f6ac4f
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Mon Apr 18 06:31:34 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I88be5c9dbb1fcc3a15592d42af7160478308b1f4

commit ca304cd08e9f8d37e4027f2f71dd77ebba3a30f9
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Fri Apr 15 17:22:44 2016 -0700

Ignore negative suffix-byte-range requests.
    
    If the client asked for "Range: bytes=--123", Swift would respond with
    a 206 and a Content-Length of -123. Now that Range header is ignored
    just like all kinds of other invalid Range headers.
    
    Change-Id: I30d4522d223076ce342d20c52f57ff0eb2aea1f4
    Closes-Bug: 1571106

commit b12222548de012ed496fba70c3d12aa31f003e3d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Apr 15 16:08:26 2016 -0700

Tighten up ResumingGetter.fast_forward
    
    When the current range is a "last N bytes" request, properly raise
    HTTPRequestedRangeNotSatisfiable/RangeAlreadyComplete errors.
    
    Also, add a couple tests for RangeAlreadyComplete with "normal" range
    requests.
    
    Change-Id: Icbfd621e7160747c5dfaf9f189da7d74e45a5347

commit 3bc33e59aaeefdb940794782a36517c2b4eb0642
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Apr 14 20:01:38 2016 -0700

Fix download resumption after getting no data.
    
    When the proxy is handling an object GET response and an object server
    fails to send data in a timely fashion, the proxy can pick up where it
    left off with another object server; other than a pause in the
    download, the client doesn't even know anything happened.
    
    However, if the proxy received the GET response headers but no data,
    it would resume at the wrong spot. In particular, for an N-byte
    object, it would ask the second object server for the last N-1 bytes
    (or equivalently, all but the first byte). For a replicated storage
    policy, this would result in the client getting an abbreviated
    download, while for an EC storage policy, the proxy would 500 after
    trying to decode a fragment set with a bogus fragment in it.
    
    This commit fixes the resumption logic to ask for all N bytes of the
    object from the second object server.
    
    Change-Id: Ib9e28c3dceaded1708e7a30844b534566c7a320c

commit 746d928a875281a7154dcd438f46a58bbf656db9
Author: Dmitriy Ukhlov <dukhlov@mirantis.com>
Date:   Fri Apr 8 16:00:16 2016 +0300

Adds eventlet monkey patching of select module if thread is pathed
    
    Oslo.messaging pika driver requires patching of select module if thread
    is patched.
    Pika driver uses select call and if it is not patched onsuming messages
    blocks whole eventlet loop
    
    Closes-Bug: #1570242
    Change-Id: I9756737309f401ebddb7475eb84725f65bca01bf

commit c96d5c671db9c96f65067d93c0a307cf4b7d91b4
Author: oshritf <oshritf@il.ibm.com>
Date:   Thu Feb 18 14:50:08 2016 +0200

Per container stat. report
    
    In addition to the container sync stat. report, keeping per container
    statistics allows administrator with more control over bytes
    transfered over a specific time per user account: The per container stats
    are crucial for billing purposes and provides the operator a 'progress
    bar' equivalent on the container's replication status.
    
    Change-Id: Ia8abcdaf53e466e8d60a957c76e32c2b2c5dc3fa

commit be84b03a07892f4972dd59309ad261fc72bd7ede
Author: dharmendra <dharmendra.kushwaha@nectechnologies.in>
Date:   Thu Apr 14 16:41:09 2016 +0530

Removing unused clause
    
    Removing unused clause from common/middleware/dlo.py
    
    Change-Id: I7de9aaefd203c4f1be00ee74b89b5184fd419469

commit fb3692c9bb662d9cadc4238920f86676857a7f1f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Apr 13 11:07:44 2016 -0700

Don't include conditional headers in SLO HEAD requests
    
    Previously, attempting to PUT a SLO manifest with `If-None-Match: *`
    would include the header when validating the segments, causing the
    upload to fail.
    
    Now when SLO validates segments, no conditional headers will be
    included in the HEAD request.
    
    Change-Id: I03ad454092d3caa73d29e6d30d8033b45bc96136
    Closes-Bug: #1569253

commit 91f980314f305c536e6e7c8607b8b52693c865f5
Author: John Dickinson <me@not.mn>
Date:   Wed Apr 13 11:57:45 2016 -0500

fix fallocate_reserve traceback
    
    Previously, fallocate_reserve could result in a traceback. The
    OSError being raised didn't have the proper errno set. This patch
    sets the errno to ENOSPC.
    
    Change-Id: I017b0584972ca8832f3b160bbcdff335ae9a1aa6

commit 0e7fca576cee81dd6ca8774760cb880c3fff9c1c
Author: Tin Lam <tl3438@att.com>
Date:   Fri Apr 8 18:17:44 2016 -0500

Convert README.md to README.rst
    
    Change-Id: I223890bd4ffe469becc2127f9362243cdb52bc08
    Closes-Bug: #1567026

commit 22933f5b55d3b8ba13b19c7004d24dfd28378f32
Author: Daisuke Morita <morita.daisuke@ntti3.com>
Date:   Mon Feb 22 18:03:48 2016 -0800

Fix bug expirer unexpectedly deletes object created after x-delete-at
    
    As reported at bug/1546067, expirer might accidentally deletes an object
    which is created after x-delete-at timestamp. This is because expirer
    sends a request with "X-Timestamp: <current_timestamp>" and tombstone
    is named as <requested_x_timestamp>.ts so if object creation time is
    between x-delete-at and expirer's DELETE request x-timestamp, the object
    might be hidden by tombstone.
    
    This possibility can be simply removed if the value of x-timestamp which
    an expirer sends is the same timestamp as x-delete-at of an actual object.
    Namely, expirer pretends to delete an object at the time an user really
    wants to delete it.
    
    Change-Id: I53e343f4e73b0b1c4ced9a3bc054541473d26cf8
    Closes-Bug: #1546067

commit b13a85367ef7f5fd64e285174859025d34b4e451
Author: Thiago da Silva <thiago@redhat.com>
Date:   Mon Dec 21 16:25:45 2015 -0200

decouple versioned writes from COPY
    
    This change removes the use of the COPY request in the versioned
    writes middleware. It changes the COPY verb for GETs and PUTs
    requests. The main reasoning for this change is to remove any
    dependency that versioning had on copy, which will allow for the COPY
    functionality to be moved to middleware and to be to the left of the
    versioned writes middleware in the proxy pipeline. In this way,
    no COPY request will ever arrive at the versioned writes middleware.
    
    A side benefit of this change is that it removes a HEAD request from
    the PUT path. Instead of checking if a current version exists, a
    GET request is sent, in case of success, a PUT is sent to the
    versions container.
    
    A unit test was removed that tested non-default storage policies.
    This test is no longer necessary, since it was used to test
    specific policy handling code in the COPY method in the proxy
    object controller.
    
    Closes-Bug: #1365862
    
    Change-Id: Idf34fa8d04ff292df7134b6d4aa94ff40887b3a4
    Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
    Co-Authored-By: Janie Richling <jrichli@us.ibm.com>
    Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit 33f06dc48f7bec2e128b44427fb429ad640cd486
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Sat Apr 9 18:47:58 2016 +0200

Fixed Sphinx errors
    
    doc/source/deployment_guide.rst:1372: ERROR: Malformed table.
    swift/obj/diskfile.py:docstring of swift.obj.diskfile.BaseDiskFileManager.yield_hashes:13: ERROR: Unexpected indentation.
    doc/source/ops_runbook/diagnose.rst:188: WARNING: Inline emphasis start-string without end-string.
    
    Change-Id: Id20eb62eb5baebb3814e7af5676badb94f17dee5

commit a829bd59770681f9d6c1ef02a6e1d5e441587a23
Author: Tin Lam <tl3438@att.com>
Date:   Fri Apr 8 18:40:30 2016 -0500

Convert CONTRIBUTING.md to CONTRIBUTING.rst
    
    Change-Id: I64c42c42db35a9f55a1df9d4ab6e97a2506b8c45
    Closes-Bug: #1567027

commit a057c409ec8a23290bc72c4fa45d55a1178f4828
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Fri Apr 8 07:02:33 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I9f4330ec20463e4d303e8ba3b67f86813a914ac5

commit d09ef0da62b64067b04a980c643f77526a9078ac
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Apr 6 15:40:42 2016 +0100

Assert that ChunkWriteTimouts are not raised
    
    Follow up for change Ibbc89449e7878fc4215e47e3f7dfe4ae58a2d638
    to add a test assertion that the ChunkWriteTimeout contexts are
    exited without raising the timeout exception in
    iter_bytes_from_response_part().
    
    Change-Id: I6d323cb26779e457fb5940093a81b349b333a0af

commit 7c0f58ec2ed020186ca3f269153b184fc02bf37a
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Thu Apr 7 07:00:08 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Ib80e3a759fa1e4a99576710607ad07fc5f259527

commit edc413b85ec2b703d7506be9c4801eb347611c58
Author: Nguyen Hung Phuong <phuongnh@vn.fujitsu.com>
Date:   Thu Apr 7 13:31:26 2016 +0700

Fix typos in Swift files
    
    Change-Id: I39dbf55c094c42347b57ef67520abff9e6fc24bc

commit 95efd3f9035ec4141e1b182516f040a59a3e5aa6
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Mar 23 13:51:47 2016 -0700

Fix infinite recursion during logging when syslog is down
    
    Change-Id: Ia9ecffc88ce43616977e141498e5ee404f2c29c4

commit 0bf518e3b0eeaf66653db6972525701cacfe6333
Author: Thiago da Silva <thiago@redhat.com>
Date:   Wed Apr 6 16:58:36 2016 -0400

remove unused current_status method
    
    Change-Id: I574919eaa14cadc800f3a1f6014221ee382ee7e0
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit 5d56f40f04fa9211a2e392d667fe395f38d2cca5
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Apr 6 11:48:48 2016 +0100

Make DirectClientException report correct ip and port
    
    When direct_client.direct_get_suffix_hashes raises a
    DirectClientException the exception message and variables
    should report the replication_ip and replication_port, as
    opposed to the ip and port values reported for all other
    case when the exception is raised.
    
    Add option to override ip and port reported in
    DirectClientException.
    
    Also adds unit tests to verify both cases.
    
    Related-Bug: 1566395
    Change-Id: If3d952847c7199f4e9f6164858085367266386d2

commit e15bceaa7e541c77f26a1f11ee2cbddbc871cbf1
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Dec 21 03:13:50 2015 -0800

Refactor CORS unit tests
    
    This is a follow-up patch for https://review.openstack.org/#/c/258392/
    That one added good unit test cases for various kinds of
    allowe_origin like '*' or ''(empty). However, the result of handling
    in Swift proxy will depend on strict_cors_mode option configuration.
    
    This patch refactors the unit tests to split out for
    strict_cors_mode = on/off and add some missing unit tests for
    each case.
    
    Change-Id: I55f7cd279436b5c9f71d81fecf06021380e35579

commit 950b601a9c9e87661e35c6ed7a97ae9611560bc5
Author: Sivasathurappan Radhakrishnan <siva.radhakrishnan@intel.com>
Date:   Tue Apr 5 22:45:17 2016 +0000

Modified REPLICATE request to use replication_ip
    
    direct_client.direct_get_suffix_hashes doesn't use replication ip and
    port for REPLICATE request. Since we have an option of doing
    replication in separate network, we can add replication_ip and port
    while creating rings if not it will get filled in with the regular
    node's ip.
    
    Change-Id: I34067df27042fc3146b795191ab8043ee1aed3ce
    Closes-Bug:1566395

commit a460e5affe7089437867afa1ed49ea677c5f4e34
Author: Thiago da Silva <thiago@redhat.com>
Date:   Mon Apr 4 16:27:14 2016 -0400

added javaswift to associated projects
    
    Change-Id: I139ac3acb1c7d2498e87df554b81824ada2dbd00
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit 4e412e0cd877be0da2b2716a301d1ae2b4c29a13
Author: Cheng Li <shcli@cn.ibm.com>
Date:   Sun Apr 3 21:07:00 2016 +0800

Fix strings with positional arguments
    
    As descriped in develop guideline,
    Any message with more than one variable should use
    named interpolation instead of positional
    http://docs.openstack.org/developer/oslo.i18n/guidelines.html#adding-variables-to-translated-messages
    
    Change-Id: Ia25517cfb52037cf49d0a6ebf528344267fb22cd
    Closes-bug:#1559431

commit ee6af69af429e1a278581531e509be92774f668d
Author: KATO Tomoyuki <kato.tomoyuki@jp.fujitsu.com>
Date:   Fri Apr 1 23:37:25 2016 +0900

[docs] Update Administrator Guide URL
    
    Change-Id: I7f6f4ffb7033d2a321362c91e05f1576847939b0

commit ebf0b220127b14bec7c05f1bc0286728f27f39d1
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Mar 30 14:19:00 2016 -0700

Fix upgrade bug in versioned_writes
    
    Previously, versioned_writes assumed that all container servers would
    always have the latest Swift code, allowing them to return reversed
    listings. This could cause the wrong version of a file to be restored
    during rolling upgrades.
    
    Now, versioned_writes will check that the listing returned is actually
    reversed. If it isn't, we will revert to getting the full (in-order)
    listing of versions and reversing it on the proxy.
    
    Change-Id: Ib53574ff71961592426cb386ef00a75eb5824def
    Closes-Bug: 1562083

commit 59bbe27fb0a40236108f09c9b3349e8faef0a95c
Author: Nguyen Hung Phuong <phuongnh@vn.fujitsu.com>
Date:   Wed Mar 30 11:07:46 2016 +0700

Fix typos in Swift files
    
    Change-Id: I34e0c9a888127704ac1910e73ddd14e27ebade13

commit 7be55acf1bc4aa07d81b30fd93e144700889898d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Feb 11 16:00:38 2016 -0800

Simplify policy-name validation slightly
    
    _validate_policy_name always either returns True or raises an exception.
    Simplify it to just being a callable that may raise an exception.
    
    Also, move the check for blank/None names into _validate_policy_name, so
    it will be applied in more cases.
    
    Change-Id: I7832a0c9c895cd75ba4c6d0e8b5568a3c8a0ea25

commit 5902015fa8495ec0ef3c1ab92ae9a34c5bda4334
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Sat Mar 26 06:35:18 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I3b5d401649fa3dea6dc43654516f7075bb06ee0d

commit 7402d7d9cf91ed00a9c86fc1511592ac2eed5d6c
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Mar 24 16:42:21 2016 -0700

Shutdown backend EC connection contexts on disconnect
    
    When eventlet.wsgi closes an ECAppIter on client disconnect we need to
    make sure our sub-iterators are also closed.  We already kill the
    backend sockets, but the executing contexts are left running until
    they timeout.  A slow client can result in needlessly holding queued
    backend fragments until the client_timeout (default 60s).
    
    Update associated test that exposed the problem to demonstrate the
    issue more quickly.
    
    Change-Id: Ibbc89449e7878fc4215e47e3f7dfe4ae58a2d638

commit 2f7d0f4a2ad2da7e6a35e5b054a47a2fafe5ed01
Author: Anh Tran <anhtt@vn.fujitsu.com>
Date:   Fri Mar 25 11:44:26 2016 +0700

Removing some redundant words
    
    This patch removes some redundant words.
    
    Change-Id: Ia79717664b06ed9a41c3c5dcf1a25e9e49e21cf2

commit 925546ae8a211b50cf7fad6634d47fd1dbfeb58e
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Fri Mar 25 06:36:40 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I6ba2f35913e6ae83607b5e268645432d455d587c

commit e97c4f794db7292a91135adb3fc3e1d6cce4dc84
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Sat Jan 16 21:59:20 2016 -0600

swift-[account|container]-info when disk is full
    
    Extended the use of the DatabaseBroker "stale_reads_ok" flag to the
    AccountBroker and ContainerBroker.  Now checks for an sqlite3 error
    from the _commit_puts call that processes the pending files.
    
    If this error is raised, then the stale_reads_ok flag will be checked
    to determine how to proceed as opposed to simply raising.
    
    The first time that print_info is attempted, the flag will be
    false, but swift-[account|container]-info will check for the
    raised exception.  If it was raised, then a warning is reported
    that the data may be stale, and another attempt will be
    made using the stale_reads_ok=True flag.
    
    Change-Id: I761526eef62327888c865d87a9caafa3e7eabab6
    Closes-Bug: 1531302

commit 3407d737c705a7afedeed0159588ab4433a601f3
Author: David Liu <david.liu@cn.ibm.com>
Date:   Thu Mar 24 16:08:19 2016 +0800

Handle tempurl Content-Disposition header missing from HEAD
    
    Content-Disposition headers should make no difference between
    GET and HEAD according to HTTP rfc.
    
    Closes-Bug: #1539805
    
    Change-Id: Ifa41a7cda2f321eb8e36420ede7912ed0a549712

commit 2f24fb9683a57b67348d65864d5af8c3a03dee67
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 23 20:49:50 2016 +0000

Check marker params in SimpleClient full listing requests
    
    Follow up for change [1] to add some assertions to check that
    marker param is included in sequential GET requests sent during
    a full listing.
    
    Extract multiple FakeConn class definitions to single class at
    module level and share between all classes.
    
    Also, explicitly unpack the return values from base request calls
    made in the full listing section of base_request, and explicitly
    return a list to make more consistent with rest of the method.
    
    [1] Change-Id: I6892390d72f70f1bc519b482d4f72603e1570163
    
    Change-Id: Iad038709f46364b8324d25ac79be4317add79df5

commit a696c1e89ef1a1c7630c14bbdda3f6c8039bb05e
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Dec 14 16:03:43 2015 +0100

Fix full_listing in internal_client
    
    The internal_client is used in swift-dispersion-report, and in case one has more
    than 10000 containers or objects these are not queried.
    
    This patch adds support to the internal_client to iterate over all
    containers/objects if the listing exceeds the default of 10000 entries and the
    argument full_listing=True is used.
    
    Closes-Bug: 1314817
    Closes-Bug: 1525995
    Change-Id: I6892390d72f70f1bc519b482d4f72603e1570163

commit 5d00ce9e3a1f5e32ae91c78b6bdfd953658ab984
Author: John Dickinson <me@not.mn>
Date:   Mon Mar 21 22:03:34 2016 -0700

2.7.0 authors and changelog updates
    
    Change-Id: I16ad0c61b048921ca01fa96862ae7eea0eec6017

commit 51bea3943f9eadb179dc7327d42dcb9062a3a8e5
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Mar 21 09:52:23 2016 +0000

Ignore files in the devices directory when auditing objects
    
    The object auditor raises an exception if there are some files in
    /srv/node (or any other defined "devices" directory). This change
    simply skips any file in the devices directory when generating
    locations for the object auditor.
    
    Change-Id: I934594994adc577799723edb6c5648685682a9e7

commit 1d03803a85ca50272071725518c7110e1b2dacb1
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Mar 15 17:09:21 2016 -0700

Auditor will clean up stale rsync tempfiles
    
    DiskFile already fills in the _ondisk_info attribute when it tries to open
    a diskfile - even if the DiskFile's fileset is not valid or deleted.
    During this process the rsync tempfiles would be discovered and logged,
    but no-one would attempt to clean them up - even if they were really old.
    
    Instead of logging and ignoring unexpected files when validate a DiskFile
    fileset we'll add unexpected files to the unexpected key in the
    _ondisk_info attribute.
    
    With a little bit of re-organization in the auditor's object_audit method
    to get things into a single return path we can add an unconditional check
    for unexpected files and remove those that are "old enough".
    
    Since the replicator will kill any rsync processes that are running longer
    than the configured rsync_timeout we know that any rsync tempfiles older
    than this can be deleted.
    
    Split unlink_older_than in common.utils into two functions to allow an
    explicit list of previously discovered paths to be passed in to avoid an
    extra listdir.  Since the getmtime handling already ignores OSError
    there's less concern of race condition where a previous discovered
    unexpected file is reaped by rsync while we're attempting to clean it up.
    
    Update some doc on the new config option.
    
    Closes-Bug: #1554005
    
    Change-Id: Id67681cb77f605e3491b8afcb9c69d769e154283

commit 125d18e0ffac572749da58fad1591d5099b5cda7
Author: OSHRITF <OSHRITF@il.ibm.com>
Date:   Wed Jan 20 15:55:30 2016 +0200

Container-Sync to perform HEAD before PUT object on remote
    
    This change adds a remote HEAD object request before each call to
    sync_row.
    Currently, container-sync-row attempts to replicate the object
    (using PUT) regardless of the existance of the object on the remote side,
    thus causing each object to be transferred on the wire several times
    (depending on the replication factor)
    
    An alternative to HEAD is to do a conditional PUT (using, 100-continue).
    However, this change is more involved and requires upgrade of both the
    client and server side clusters to work.
    In the Tokyo design summit it was decided to start with the HEAD approach.
    
    Change-Id: I60d982dd2cc79a0f13b0924507cd03d7f9c9d70b
    Closes-Bug: #1277223

commit eecb1f2d54b8737c784952072b7374ceb7647245
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Wed Mar 23 06:16:13 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Ia2c2819db372da46538d71a80888a4e27538bdcd

commit 2afa3681cb9c8a74cbe23b1de8d4f2e1a06e8f3a
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Mar 22 11:51:11 2016 +0000

Add .eggs/* to .gitignore
    
    After running:
    
      python setup.py build_sphinx
    
    there is a .eggs directory left in the repo root directory
    which is not currently ignored by git.
    
    Change-Id: Id15811f94046fd8bb22153425bf5cafe6c045453

commit 736de613f1e2a6a460f4499cbd1e9f7f1fc3da68
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Mar 22 11:36:32 2016 +0000

Docs: Container sync does not require POST-as-COPY
    
    Updates docs to remove warnings that container sync only
    works with object_post_as_copy=True. Since commit e91de49
    container sync will also sync POST updates when using
    object_post_as_copy=False.
    
    Change-Id: I5cc3cc6e8f9ba2fef6f896f2b11d2a4e06825f7f

commit 88b575895750c2c00f75a6193cc31bc930e980fd
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Mar 22 11:10:41 2016 +0000

Remove unused code from container sync
    
    Change-Id: Ia44138aadcd30c474f744a9c552220e18302ecc6

commit 365276464ce41070f14fee4314e665dd2934bfa6
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Tue Mar 22 06:16:34 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I70db7d29a9859cb47144ac49df8c289d1c2ec3e6

commit fd86d5a95d73714365c07cb36bfd1404306142a7
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Feb 15 19:17:01 2016 +0000

Skip already checked partitions when auditing objects after a restart
    
    The object auditor will save a short status file on each device, containing a
    list of remaining partitions for auditing. If the auditor is restarted, it will
    only audit partitions not yet checked. If all partitions on the current device
    have been checked, it will simply skip this device. Once all partitions on all
    disks are successfully audited, all status files are removed.
    
    Closes-Bug: #1183656
    
    Change-Id: Icf1d920d0942ce48f1d3d374ea4d63dbc29ea464

commit b685e8513525569ec1e0e4c34377866b6f5c1f7e
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Mon Mar 21 06:16:52 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I3cae432fdcf311bc3654956a1f61ce22ecae0933

commit 0ad1b0cca1f2538ac8fe3ef27d9e06c102157000
Author: oshritf <oshritf@il.ibm.com>
Date:   Thu Mar 17 16:53:04 2016 +0200

Container sync nodes shuffle cleanup
    
    Since commit "Update container sync to use internal client" get_object is
    done using internal_client and not directly on nodes which makes the block
    of code to shuffle the nodes redundant.
    
    Change-Id: I45a6dab05f6f87510cf73102b1ed191238209efe

commit f99b785f47f458baa6a5286e6f6e942a6e2745f3
Author: Andreas Jaeger <aj@suse.com>
Date:   Fri Mar 18 21:30:58 2016 +0100

Avoid translation problem
    
    Today's import failed again for zh-TW since the translator of a string
    changed the order of the entries and then msgfmt complained with
    "format specifications in 'msgid' and 'msgstr' for argument 1 are not
    the same"
    
    Use named arguments to allow translators to switch the order of the
    entries.
    
    Logfile:
    https://jenkins.openstack.org/job/swift-propose-translation-update/179/console
    
    Change-Id: Ide7480a8293a8439ce89d0ad22be74f1f0ad930c

commit e1f12960884f9ea52416891a8246a53f9673348b
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Tue Jan 5 14:40:50 2016 -0600

Re-format the SLO manifest file on new multipart-manifest GET call
    
    Currently, the multipart-manifest=get call returns output in json
    format that is inconsistent with the format that is used for the
    multipart-manifest=put.
    
    This in turn introduces a new call: ?multipart-manifest=get&format=raw
    
    Change-Id: I2242943a738f667cbda6363bcb6a017f341e834f
    Closes-Bug: 1252482

commit a85149c40e34b3ccd2aa5f3865bb85d2378d6086
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Thu Mar 17 22:12:42 2016 +0000

Update ring overview for reuse of device ids
    
    Change-Id: Ied15706d4cc931b5314001a02457f83e175c5d24

commit c70abba52958a78d21c00a08488d1c2a7c3df554
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Mar 17 11:55:14 2016 -0700

Adjust replica count before pulling parts from failed devices
    
    When your device count falls below your replica count you can either add
    devices or reduce the replica count.
    
    Trying to reduce your replica count fails about half the time because
    removing parts from from failed devices temporarily invalidates your
    _replica2part2dev table with NONE_DEV which can result in an IndexError
    in _adjust_replica2part2dev_size.
    
    If you adjust the replica count first you won't have to worry about
    tracking unassigned parts from failed devices.
    
    Closes-Bug: #1558751
    
    Change-Id: I99dc776fd260a2ba68ca77d7b5ed5120d10b06de

commit 9f7e947dca83a8b7c5e81493b88669472ef2875d
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Thu Mar 17 14:45:42 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I95466266e314cf4a564c7f93cfae7df9df21b60c

commit 12dd408823df158359e99fb01716f2059140c5c9
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 16 17:41:30 2016 +0000

Put correct Etag and Accept-Ranges in EC 304 and 416 responses
    
    When using an EC policy, 304 responses to conditional GETs
    are missing the Accept-Ranges header and have the wrong ETag
    value. 412 responses also have the wrong etag.
    
    416 responses to ranged GETs also have the wrong ETag.
    
    This patch ensures behaviour with EC policy is consistent
    with replication policy:
    
      - 304 and 416 responses have correct etag and Accept-Ranges
      - 412 responses have correct Etag but no Accept-Ranges
    
    Co-Authored-By: Mahati Chamarthy <mahati.chamarthy@gmail.com>
    Co-Authored-By: Thiago da Silva <thiago@redhat.com>
    
    Closes-Bug: #1496234
    Closes-Bug: #1558197
    Closes-Bug: #1558193
    Change-Id: Ic21317b9e4f632f0751133a3383eb5487379e11f

commit 6ea7635f87ce480e712439bf20e0e4d9ca81b352
Author: Charles Hsu <charles0126@gmail.com>
Date:   Thu Dec 31 12:17:21 2015 +0800

Add a note for functional tests with Keystone.
    
    Change-Id: I9da8f6a56ffb06e89b8da8c06b26ebc8452b652d
    Closes-Bug: #1530254

commit 6efee0ebb12a8d0c36aad83d4fd592c4e51c5722
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 16 11:38:33 2016 +0000

Make keystone middleware options consistent in docs
    
    Bring overview_auth.rst and proxy server man page
    up to date with changes made in [1]
    
    [1] Change-Id: I373734933189c87c4094203b0752dd3762689034
    
    Change-Id: Ia16f0c391e7c357ccb9c13945839dc5647e49a13

commit ecbcc94989a385efd1a7c60e7c91e3e8898c5b31
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu Mar 10 06:42:57 2016 -0800

Fix ssync related object-server docs
    
    Swift now uses SSYNC verb instead of old REPLICATION verb for ssync
    protocol. This patch replaces all docs written as REPLICATION into
    SSYNC and fix a few words for explanation.
    
    Change-Id: I1253210d4f49749e7d425d6252dd262b650d9548

commit f595a7e70492c2751d8c6f5ab60b5512e63281cf
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Fri Aug 29 17:14:46 2014 +1000

Add concurrent reads option to proxy
    
    This change adds 2 new parameters to enable and control concurrent GETs
    in swift, these are 'concurrent_gets' and 'concurrency_timeout'.
    
    'concurrent_gets' allows you to turn on or off concurrent GETs, when
    on it will set the GET/HEAD concurrency to replica count. And in the
    case of EC HEADs it will set it to ndata.
    The proxy will then serve only the first valid source to respond.
    This applies to all account, container and object GETs except
    for EC. For EC only HEAD requests are effected.
    
    It achieves this by changing the request sending mechanism to using
    GreenAsyncPile and green threads with a time out between each
    request.
    
    'concurrency_timeout' is related to concurrent_gets. And is the
    amount of time to wait before firing the next thread. A value of 0
    will fire at the same time (fully concurrent), setting another value
    will stagger the firing allowing you the ability to give a node a
    shorter chance to respond before firing the next. This value is a float
    and should be somewhere between 0 and node_timeout. The default is
    conn_timeout. Meaning by default it will stagger the firing.
    
    DocImpact
    Implements: blueprint concurrent-reads
    Change-Id: I789d39472ec48b22415ff9d9821b1eefab7da867

commit f87a5487b5224f77261d82a1087d31820c29e8f8
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Mar 14 16:52:50 2016 -0700

Make rsync ignore it's own temporary files
    
    In situations where rsync may inadvertently be unable to cleanup it's
    temporary files we shouldn't spread them around the cluster.
    
    By asking our rsync subexec to --exclude patterns that match it's own
    convention for temporary naming we'll only ever transfer real replicated
    artifacts and never temporary artifacts which should always be ignored
    until they are fully transfered.
    
    Cleanup of stale rsync droppings should be performed by the auditor and
    will be addressed in a separate change related to lp bug #1554005.
    
    Closes-Bug: #1553995
    
    Change-Id: Ibe598b339af024d05e4d89c34d696e972d8189ff

commit 9e90d309621584be173b9be7ac91c469a013ab96
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Mon Mar 14 06:13:37 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I30d06a1d96d4bd0e2e4972cb8ede7ebc08beae2d

commit a537684c77a2e91c0cc6e4c4953a7e993f6a7005
Author: Brian Cline <bcline@softlayer.com>
Date:   Mon Mar 14 00:17:47 2016 -0500

Don't report recon mount/usage status on files
    
    Today recon will include normal files in the payload it returns for
    /recon/unmounted and /recon/diskusage. As a result it can trigger
    bogus alarms on any operations-side monitoring checking for unmounted
    disks or disks that show up in diskusage with weird looking stats.
    
    This change adds an isdir check for the entries it finds in /srv/node.
    
    Change-Id: Iad72e03fdda11ff600b81b4c5d58020cc4b9048e
    Closes-bug: #1556747

commit 1cef9f0e3375a9113985fe60b80650395ffc96ad
Author: zhangguoqing <zhang.guoqing@99cloud.net>
Date:   Sun Mar 13 10:51:18 2016 +0000

Fix py34 error of indexing 'dict_keys' object
    
    In python3, keys() of a dict isn't a list and cannot be indexed.
    
    Change-Id: I8a5b8e66a69f7096b2e5388bdeb12bb800879158
    Closes-Bug: #1550184

commit 87fe86bcf8f061d91a8587e90613eee4e83c41d2
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Sat Mar 12 06:12:49 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Iee254bef3513111d339cafb4cb752eb4ad4c3b03

commit 3ac7cab32bcd08e032ad587a5982bdc997fc20c1
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 9 10:17:12 2016 +0000

Remove reference to py26 in tox testing instructions
    
    Also change some rst formatting and clarify that tox
    is used to run more than just *unit* tests.
    
    Change-Id: I263af7b4f7a554da3a391cf5104af573c89e3825

commit 8aca4e8eb36c67038335462abdb9a58ee5263b5c
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Fri Mar 11 06:22:54 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Ic39cc9ffb582909da3a827243b628206de5301a0

commit 0bc5a69d416004924bd17c14d8455f65433751bd
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Mar 10 18:02:44 2016 +0000

Wait on greenthreads before unmocking http_connect
    
    The fake_spawn() context manager wraps spawn and waits
    for greenthreads to complete (such as the async_update threads).
    The wait needs to be done before http_connect is un-mocked, so
    the fake_spawn context manager should exit *before* any context
    manager that mocks the http_connect method.
    
    Also add fake_spawn to _test_PUT_then_POST_async_pendings()
    
    Related-Bug: #1536376
    Related-Bug: #1514111
    Closes-Bug: #1555739
    
    Change-Id: I15f36e191cfe3ee6c82b4be56e8618ec0230e328

commit e38b53393fee311ca4e351b6d9ee3d2bf02441a4
Author: Donagh McCabe <donagh.mccabe@hp.com>
Date:   Wed Mar 9 14:28:17 2016 +0000

Cleanup of Swift Ops Runbook
    
    This patch cleans up some rough edges that were left (due to
    time constraints) in the original commit.
    
    Change-Id: Id4480be8dc1b5c920c19988cb89ca8b60ace91b4
    Co-Authored-By: Gerry Drudy gerry.drudy@hpe.com

commit 994fa7b115f31da37d064a60a823d1c87545d292
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Mar 9 16:49:20 2016 +0000

Fix object server test not using correct policy
    
    Without the correct request header, the test that should be
    using an EC policy was in fact using a replication policy.
    
    Change-Id: Id44d0d615a4fd09aadfb286425939aea6abdf7b0

commit f581fccf71034818d19062593eeb52a4347bb174
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Feb 9 15:59:52 2016 -0800

By default, disallow inbound X-Timestamp headers
    
    With the X-Timestamp validation added in commit e619411, end users
    could upload objects with
    
        X-Timestamp: 9999999999.99999_ffffffffffffffff
    
    (the maximum value) and Swift would be unable to delete them.
    
    Now, inbound X-Timestamp headers will be moved to
    X-Backend-Inbound-X-Timestamp, effectively rendering them harmless.
    
    The primary reason to allow X-Timestamp before was to prevent
    Last-Modified changes for objects coming from either:
    
      * container_sync or
      * a migration from another storage system.
    
    To enable the former use-case, the container_sync middleware will now
    translate X-Backend-Inbound-X-Timestamp headers back to X-Timestamp
    after verifying the request.
    
    Additionally, a new option is added to the gatekeeper filter config:
    
        # shunt_inbound_x_timestamp = true
    
    To enable the latter use-case (or any other use-case not mentioned), set
    this to false.
    
    Upgrade Consideration
    =====================
    
    If your cluster workload requires that clients be allowed to specify
    objects' X-Timestamp values, disable the shunt_inbound_x_timestamp
    option before upgrading.
    
    UpgradeImpact
    Change-Id: I8799d5eb2ae9d795ba358bb422f69c70ee8ebd2c

commit dd2fbcd4c75974859820f76c3aa78be445170202
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Wed Mar 9 06:35:57 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I9196deac606a3d073fb1c400cbee778cbbbd1c1e

commit 3eeff7c0fc79fab594e3bbb04be74d397c920253
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Mar 8 15:23:40 2016 -0800

Fix misleading comment
    
    Change-Id: I2e0e671a08fb855bf53c57987c08a7eefca7078a

commit dc6db66d5e4423ee39e8bf423916c035e55732d0
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Mar 8 12:32:39 2016 -0800

Spot check more hashes for increase_part_power
    
    Just a touch of cleanup to the unittest to demonstrate the stability of the
    guarantee that objects will hash to the same nodes but have different parts.
    
    Change-Id: I4a24187755455366a6435816a138f6175ae713a4

commit 9db7391e55e069d82f780c4372ffa32ef4e79c35
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Mon Mar 7 18:42:01 2016 +0000

Object POST update gets distinct async pending file
    
    Each object update to a container server is saved in a
    pending file if the initial update attempt fails. Pending
    file names were derived from the update request's x-timestamp,
    which is equal to the object's data file timestamp. This meant
    that updates due to an object POST used the same async pending
    file as updates due to the object's PUT.
    
    This is not so bad because the object POST update has a superset
    of the metadata included in the PUT update. But there is a risk
    of a race condition causing an update to be lost: the updater may
    open an update file due to a PUT whuile the object server is
    writing an update due to a POST to the same file name. The
    updater could then unlink the file before the more recent update
    for the POST is sent.
    
    This patch changes the POST update pending file name to be derived
    from the object's metadata timestamp, thus making it distinct from
    the PUT update pending file name. There is no upgrade impact since
    existing pending files will continue to be processed.
    
    Change-Id: I1b093c837efe8c2a64e92075ebd5e1b93e30efb9

commit 82c8023b894aa7a1fe12fbf1efbfad47afdf86b3
Author: Mahati Chamarthy <mahati.chamarthy@gmail.com>
Date:   Tue Mar 8 19:01:53 2016 +0530

remove hash_cleanup_listdir legacy translation method
    
    Change-Id: I0b96dfde32b4c666eebda6e88228516dd693ef92
    closes-bug:#1550569

commit 9430f4c9f5e79026f7275f8ec32ef4a180c5e8fc
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Mar 2 10:28:51 2016 +0000

Move HeaderKeyDict to avoid an inline import
    
    There was a function in swift.common.utils that was importing
    swob.HeaderKeyDict at call time. It couldn't import it at compilation
    time since utils can't import from swob or else it blows up with a
    circular import error.
    
    This commit just moves HeaderKeyDict into swift.common.header_key_dict
    so that we can remove the inline import.
    
    Change-Id: I656fde8cc2e125327c26c589cf1045cb81ffc7e5

commit fa0c5f244f907ed2d5bd6d90f0be44869d7d1de6
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Feb 29 13:14:59 2016 +0000

Additionally break container servers during fast-post probes
    
    The existing probetests were already doing brainsplitting on the object
    servers in order to introduce composite metdata timestamp reconciliation.
    
    But they were not introducing failures at the container server level, and
    therefore not covering functional testing of those servers replication
    consistency repair.
    
    This change adds a brain splitter for the container servers to additionally
    exercise the consistency engine at that layer under the existing probetest so
    that you can observe their correct behavior while exercising the probetests
    [1].
    
    1. I used a script like this to watch the object servers and container db's
    repair themselves -> https://gist.github.com/clayg/a8077d9d29ff68bb0edd
    
    Change-Id: Ic5bb03dffef7e30c58338cef969a105577b84620

commit be54d0c928528cdc1b12e1bcb1614ea8859fae2e
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Mon Dec 7 21:45:43 2015 +0530

clear pycache and remove all pyc/pyo before starting unit test
    
    Delete python bytecode before every test run.
    Because python creates pyc files during tox runs, certain
    changes in the tree, like deletes of files, or switching
    branches, can create spurious errors.
    
    Closes-Bug: #1368661
    Change-Id: Iedcb400fa3b0417f5bb8e943b17758fcfb4070c6

commit 936881543e60fa437cd0f91f029f9874fd7437d9
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Jan 14 18:31:21 2016 +0000

Reclaim isolated .meta files
    
    It is possible for an object dir to contain only
    a .meta file*, and currently this file will never
    be deleted. This patch changes the diskfile on disk
    file processing to identify any isolated meta files
    and mark them as ready to be reclaimed, so that the
    cleanup will reclaim them if they are older than
    reclaim age.
    
    *An isolated .meta can occur as follows:
    
    Consider two replicas of the same object whose ondisk files
    have diverged due to failures:
    
      A has t2.ts
      B has t1.data, t4.meta
    
    (The DELETE at t2 did not make it to B. The POST at t4 was
    rejected by A.)
    
    After ssync replication the two ondisk file sets will be:
    
      A has t2.ts
      B has t2.ts, t4.meta
    
    Once t2 becomes older than (current time - reclaim age) we are
    left with:
    
      B has t4.meta
    
    Closes-Bug: #1534685
    Change-Id: I3a175fd948a331d08df1b28f9831d7529216c6f1

commit d456d9e934d8eb845d83b1e0c29247aac87e2e92
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Oct 7 10:17:39 2015 +0100

Don't ssync data when only a durable is missing
    
    If an EC diskfile is missing its .durable file (for example
    due to a partial PUT failure) then the ssync missing check
    will fail to open the file and will consider it
    missing. This can result in possible reconstruction of the
    fragment archive (for a sync job) and definite transmission
    of the fragment archive (for sync and revert jobs), which is
    wasteful.
    
    This patch makes the ssync receiver inspect the diskfile
    state after attempting to open it, and if fragments exist at
    the timestamp of the sender's diskfile, but a .durable file
    is missing, then the receiver will commit the diskfile at
    the sender's timestamp. As a result, there is no longer any
    need to send a fragment archive.
    
    Change-Id: I4766864fcc0a3553976e8fd85bbb2fc782f04abd

commit 3ff94cb785867382ff6c37cb256d1b0f5381abaa
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Feb 15 19:16:08 2016 +0000

Add internal method to increase ring partition power
    
    This method increases the partition power of an existing ring by one. It does
    not move any data nor does it exposes a CLI command yet; it is only intended to
    be used in a future version to do the actual ring modification itself.
    
    An existing object that is currently located on partition X will be placed
    either on partition 2*X or 2*X+1 after the partition power got increased. The
    reason for this is the Ring.get_part() method, that does a bitwise shift to the
    right.
    
    To avoid actual data movement to different disks or even nodes, the allocation
    of partitions to nodes needs to be changed. The allocation is pairwise due to
    the above mentioned new partition scheme. Therefore devices are allocated like
    this, with the partition being the index and the value being the device id:
    
    OLD: 0,    3,    7,    5,    2,    1,    ...
    NEW: 0, 0, 3, 3, 7, 7, 5, 5, 2, 2, 1, 1, ...
    
    If an operator stops the cluster, increases the partition power and renames &
    hardlinks the existing data it is possible to do a power shift without actually
    moving data. Please see the partition power spec for further details on this.
    
    Change-Id: I063fd8077497ee8c14d9065f07b4ec0fb5cbe180
    Partially-Implements: spec increasing_partition_power

commit 6b4e73bb15c680ada8b30279b090c6123ce5731f
Author: Charles Hsu <charles0126@gmail.com>
Date:   Thu Feb 25 13:31:51 2016 +0800

Fix account-reaper unable to delete all containers.
    
    When reaper try to find the correct container_shard number,
    but it doesn't compare local device's name, so it always
    return first entry's index of local devices from the ring
    to container_shard.
    That causes some containers will be skip and objects under
    these containers won't be delete.
    
    Change-Id: I0e2bbdd99add86ee46e856920c0740c4aa13c77d
    Closes-Bug: #1549615

commit 22685d6231003bd236e7fc363b58e86f35ab80bb
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Mar 1 13:54:09 2016 +0000

Faster suffix invalidations on object PUT/DELETE
    
    Before this commit, we performed a full read-modify-write cycle on a
    partition's hashes.pkl to invalidate the relevant suffix whenever an
    object PUT or DELETE completes.
    
    Now we append invalid suffixes to a new file, "hashes.invalid",
    alongside hashes.pkl. When we actually get a REPLICATE request and
    need to compute the hashes, *then* we perform a full read-modify-write
    on hashes.pkl and clear out hashes.invalid.
    
    Change-Id: Ia7add438e25688b4b286f0110a4e43490e11ad75

commit 6d8be59fce17008369656628615efcd2677dfe36
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Feb 22 22:41:51 2016 -0800

Fix ringbuilder tests
    
    Some of tests in test/unit/cli/test_ringbuilder doesn't assert
    the exit code and unfortunately some of these passed even if the
    statement fails for the assertion actually.
    
    This patch enables to assert the exit code from ringbuider and
    fixes some code/test bugs I noticed.
    
    Change-Id: I18fa675ba8a90678e2b5ccb5f90eafab01d22787

commit fe70898daced5f5fd698dd27098451439fdd8d08
Author: Tim Burke <tim.burke@gmail.com>
Date:   Sun Feb 28 01:18:07 2016 +0000

Require account/container metadata be UTF-8
    
    Otherwise, we get a UnicodeDecodeError when we call json.dumps()
    
    Change-Id: Ie200d029e1fd7f0ff0956c8ced98207e11ef9080

commit 0628f1268c804aa555ff3873d11831266c1b1cb1
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Fri Oct 23 15:50:21 2015 -0500

Add unit tests for swift.account.reaper
    
    This patch adds more unit tests to diminish missing pieces
    of the coverage in the account_reaper unit test.
    
    Change-Id: Ib9e875ddd1334a4a67037dcfbd42d3b008ccd4e7

commit 2d73dd9e660b50ef04a20012963a15376983923f
Author: Cheng Li <shcli@cn.ibm.com>
Date:   Sat Feb 27 23:26:47 2016 +0800

keep etag of target source instead of possible_source
    
    In proxy/controllers/base.py : ResumingGetter._get_source_and_node
    self.used_source_etag should be etag of target source
    instead of possible_source.
    
    Change-Id: Id6c33e79f65cb4ad9e3249957dd57dc2da8cad98
    Closes-bug: #1550734

commit 75ab3cb78862647644af7eaf5e0fb590e73ae341
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Oct 14 15:49:35 2015 +0100

Stop staticweb revealing container existence to unauth'd requests
    
    When a container has `X-Container-Meta-Web-Listings: false` then
    staticweb will return a 404 in response to a GET or HEAD on the
    container, regardless of whether the request is auth'd. That provides
    a way to probe for container existence. It should return a 401 if the
    request is not auth'd.
    
    This patch adds a call to swift.authorize before returning the 404.
    
    Closes-Bug: 1506116
    Change-Id: I382323b49dc8f6d67bf4494db7084a860a10db59

commit b5a243e75a8033988063d2c1c90ac373bc0050d2
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Fri Feb 19 18:07:18 2016 -0600

Set backend content length for fallocate - EC Policy
    
    Currently, the ECObjectController removes the 'content-length' header.
    This part is ok, except that value is being used to set
    'X-Backend-Obj-Content-Length', so it is always 0.  This leads to not
    calling fallocate (details on bug) on a PUT since the size is 0.
    
    This change makes use of some numbers returned from the EC Driver
    get_segment_info method in order to calculate the expected on-disk
    size that should be allocated.  The EC controller will now set the
    'X-Backend-Obj-Content-Length' value appropriately.
    
    Co-Authored-By: Kota Tsuyuzaki
    Co-Authored-By: John Dickinson
    Co-Authored-By: Tim Burke
    Change-Id: Ifd16c1438539e6fd9bb2dbcd053d11bea2e09fee
    Fixes: bug 1532008

commit 7aaca029175661d45d05d41cce367c0ae19cab41
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Feb 11 13:18:07 2016 +0100

Add comments to role-based access control functional tests
    
    Change-Id: Ic18c8a2abc33e2f175f948991af0189935b8702f

commit 335d58611d0daf87783e565b521974b456f5c636
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Tue Dec 15 22:42:18 2015 +0100

Keystone middleware deprecated option is_admin removed
    
    It has been deprecated from Swift 1.8.0 (Grizzly)
    
    Change-Id: Id6bc10c3e84262c0a9e6160a76af03c0ad363e9c

commit ceaaedbe4057542e374200673dd323ba64bb42eb
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Mar 11 11:22:53 2015 +0000

Make functional tests tolerate pre-existing versioned container
    
    test/functional/tests.py:TestAccount.testAccountHead relies
    on the account having a known number of containers. The test
    setup attempts to delete all existing containers but this fails
    a container has versions (and so is not emptied, and cannot be
    deleted). The tests then fails because the expected number of
    containers does not match the actual.
    
    'bin/resetswift' before running tests will obviously clear all
    state but is not always convenient.
    
    This change removes any x-versions-location header before deleting
    containers during test setUp.
    
    Steps to recreate the pre-condition for failure on master:
    (based on original work by clayg)
    
    swift post target -r '.r:*, .rlistings'
    swift post source -H 'x-versions-location: target'
    for i in {1..4}; do
      echo "junk${i}" > junk
      swift upload source junk
    done
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Change-Id: I3efb6c20dc1fb3e979087e8a93d04ba7e346b5b6

commit 432e280aef1ba08bbb8dc239260604024325c9f6
Author: Timur Alperovich <timur.alperovich@gmail.com>
Date:   Wed Jul 15 14:22:45 2015 -0700

Correctly handle keys starting with the delimiter.
    
    When processing keys where the names start with the delimiter
    character, swift should list only the delimiter character. To get the
    list of nested keys, the caller should also supply the prefix which is
    equal to the delimiter.
    
    Added a functional test and unit tests to verify this behavior.
    
    Fixes Bug: 1475018
    
    Change-Id: I27701a31bfa22842c272b7781738e8c546b82cbc

commit 04ff0927eeda0cc3ae2409c6fe6c5a9b6e39fdd2
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon Aug 24 15:41:23 2015 -0700

Make the object auditor's run-once mode run once.
    
    If you invoked the object auditor with --once, it would run the
    full-audit checker(s) once, but it would run the ZBF checker over and
    over until the full-audit checkers were done. Now it runs the ZBF and
    full-audit checkers once each.
    
    Change-Id: Ieeaa6fba4184a069756ee150727f24df7833697a

tags:

added: in-feature-hummingbird

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-06-01: Fix proposed to swift (feature/crypto)

#7

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/323875

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-06-01: Fix merged to swift (feature/crypto)

#8

Download full text (4.8 KiB)

Reviewed: https://review.openstack.org/323875
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=03b8b4bfa9e28076ece78f131c95655b38b89971
Submitter: Jenkins
Branch: feature/crypto

commit 86e9e827bab7bedb529a467f217eb17b3b7e0967
Author: John Dickinson <email address hidden>
Date: Tue May 31 11:27:43 2016 -0700

add explicit HA info to the deployment guide

Change-Id: I7614952c523080fe50eaf839b54a8064439817ce

commit 5fe392b562de3baed080704df433fb392cb4fb31
Author: Ondřej Nový <email address hidden>
Date: Tue May 31 16:25:50 2016 +0200

Fixed typo

Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708

commit a821dd42de1762e61a26ec9cac90976a9cd4e51f
Author: Tim Burke <email address hidden>
Date: Thu May 26 13:42:12 2016 -0700

Don't include holes when reporting how many devices a ring has

Change-Id: I9b933051aec009c6108ee9d2dd5c0978772bf699

commit b52eccb3b1ea0591f0040587228d3705b5d3f68d
Author: Clay Gerrard <email address hidden>
Date: Wed May 25 11:21:25 2016 -0700

Clarify overload best practices in admin guide

Change-Id: Ib7c08bdeab6374771bb8e2b05053e7e16973524d

commit f1fd50723bb84c4941e949895576733f6eb67793
Author: Christian Schwede <email address hidden>
Date: Wed May 25 09:53:31 2016 +0200

Add dispersion --verbose example to admin guide

Change-Id: I5f9cacedde2a329332ccf744800b6f2453e8b28e

commit b3ab715c055283ccfea9a504d6da20741d82e7ad
Author: Matthew Oliver <email address hidden>
Date: Wed May 25 14:35:54 2016 +1000

Add ring-builder dispersion command to admin guide

    This change updates the admin guide to point out the dispersion command
    in swift-ring-builder and mentions the dispersion verbose table to make
    it more obvious to operators.

Change-Id: I72b4c8b2d718e6063de0fdabbaf4f2b73694e0a4

commit fb7a8e9ab7596a36a6992a3a8f8c6d005a2c2829
Author: Tim Burke <email address hidden>
Date: Tue May 24 13:37:58 2016 -0700

Add links to mitaka install guides

Change-Id: I62331923751c521daded4468b5cc5f03655226bc

commit e09c4ee7800e82aa09ca2f6ae375420b766182a4
Author: Tim Burke <email address hidden>
Date: Fri Apr 29 12:12:00 2016 -0500

Allow concurrent bulk deletes

    Before, server-side deletes of static large objects could take a long
    time to complete since the proxy would wait for a response to each
    segment DELETE before starting the next DELETE request.

    Now, operators can configure a concurrency factor for the slo and bulk
    middlewares to allow up to N concurrent DELETE requests. By default, two
    DELETE requests will be allowed at a time.

    Note that objects and containers are now deleted in separate passes, to
    reduce the likelihood of 409 Conflict responses when deleting
    containers.

    Upgrade Consideration
    =====================
    If operators have enabled the bulk or slo middlewares and would like to
    preserve the prior (single-threaded) DELETE behavior, they must add the
    following line to their [filter:slo] and [filter:bulk] proxy config
    sections:

delete_concurrency = 1

This may be done prior to upgrading Swif...

OpenStack Object Storage (swift)

post-as-copy or COPY of SLO manifest results in wrong content-type in container listing

Bug Description

Other bug subscribers

Remote bug watches