Bug #1457691 “node timeout on overwrite can easily cause mis-mat...” : Bugs : OpenStack Object Storage (swift)

John Dickinson (notmyname) on 2015-06-03

tags:

added: ec

Revision history for this message

Bill Huber (wbhuber) wrote on 2015-06-04:

#1

Investigating this bug.

Revision history for this message

paul luse (paul-e-luse) wrote on 2015-06-04:

#2

FYI we are not feature complete yet wrt GET handling of previous partial PUTs. Feel free to investigate, will share design thoughts on general handling of the incomplete scenarios next week sometime. This sounds like it might be covered by pending implementation

Bill Huber (wbhuber) on 2015-06-24

Changed in swift:
assignee:	nobody → Bill Huber (wbhuber)

clayg (clay-gerrard) on 2015-06-25

Changed in swift:
importance:	Undecided → High

Revision history for this message

clayg (clay-gerrard) wrote on 2015-06-26:

#3

failure reproduction would include setting up a probe test like this:

ec 4+2 scheme - 8 devices

             p0 p1 p2 p3 p4 p5 p6 h0 h1 h2 h3
    t0 PUT t0#0 t0#1 X t0#3 X t0#5 t0#6 t0#2 t0#4
    t1 PUT t1#0 X t1#2 X t1#4 t1#5 t1#6 t1#1 t1#3
        GET t1#0 t0#1 t1#2 t0#3 t1#4

Even better would be a unittest that would let us describe a list of a *variety* of backend 404s/timestamp/frag-index'es responses and validate proxy responses...

Revision history for this message

clayg (clay-gerrard) wrote on 2015-06-26:

#4

https://gist.githubusercontent.com/clayg/40516dfc974d2325f85d/raw/e50433c181001775567922258086734a5d4d0699/gistfile1.txt

I guess I don't know how to do fixed width in lp

Changed in swift:
status:	New → Confirmed

Revision history for this message

clayg (clay-gerrard) wrote on 2015-06-26:

#5

Related failure condition is when all of the fragments are *available* but some of the object servers think they are not *durable* (lp bug #1469094)

Revision history for this message

clayg (clay-gerrard) wrote on 2015-06-30:

#6

failing probetest for ec overwrite Edit (6.2 KiB, text/plain)

One of the simplest reproducible failure conditions is an overwrite the goes to a handoff.

If one fragment from an old version of the object manages to stick around and get picked up on the subsequent GET the whole request will fail - even though asking for another fragment from almost *any* node would have returned enough fragments to rebuild the object.

Probetest attached.

Revision history for this message

paul luse (paul-e-luse) wrote on 2015-07-13:

#7

FYI this WIP patch will be related for sure... https://review.openstack.org/#/c/201283/

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-07-14: Fix proposed to swift (master)

#8

Fix proposed to branch: master
Review: https://review.openstack.org/201791

Changed in swift:
status:	Confirmed → In Progress

Revision history for this message

paul luse (paul-e-luse) wrote on 2015-07-17: Re: Insufficient Fragments occurred on EC GET object

#9

So I'm going to fix this over on the "support alternate FI patch" at https://review.openstack.org/#/c/201283/ because the fix changes that patch considerably just simply by means of where its at. The problem is that the current logic requires that *all* FA's fetched have matching ETAGs even if there's enough of one set of matching ETAGs to rebuild.

Revision history for this message

paul luse (paul-e-luse) wrote on 2015-07-18: Re: EC GET w/not work with just one PUT during handoff

#10

reworded title slightly to hopefully show how easy it is to hit this bug. Fix coming this weekend....

summary:

- Insufficient Fragments occurred on EC GET object
+ EC GET w/not work with just one PUT during handoff

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-07-20: Change abandoned on swift (master)

#11

Change abandoned by Bill Huber (<email address hidden>) on branch: master
Review: https://review.openstack.org/201791
Reason: peluse has been working on the fix for the root cause with the probe test this patch included. No need to do duplicates and going to close/abandon this one.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-07-30:

#12

Change abandoned by paul luse (<email address hidden>) on branch: master
Review: https://review.openstack.org/201283
Reason: moved over to https://review.openstack.org/#/c/207165/ to pick up a dependency

paul luse (paul-e-luse) on 2015-08-04

Changed in swift:
assignee:	Bill Huber (wbhuber) → paul luse (paul-e-luse)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-12: Fix proposed to swift (master)

#13

Fix proposed to branch: master
Review: https://review.openstack.org/212187

clayg (clay-gerrard) on 2015-08-13

summary:

- EC GET w/not work with just one PUT during handoff
+ node timeout on overwrite can easily cause mis-matched etag fragment to
+ 503

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-14: Change abandoned on swift (master)

#14

Change abandoned by paul luse (<email address hidden>) on branch: master
Review: https://review.openstack.org/207165
Reason: functionality shuffled off into:
https://review.openstack.org/#/c/212187/
https://review.openstack.org/#/c/213147/

paul luse (paul-e-luse) on 2015-08-25

Changed in swift:
importance:	High → Critical

Revision history for this message

paul luse (paul-e-luse) wrote on 2015-08-31:

#15

critical for EC, does not affect Swift in general... moved back to high

Changed in swift:
importance:	Critical → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-02: Fix merged to swift (master)

#16

Reviewed: https://review.openstack.org/212187
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=893f30c61d280804e417790dd34ba7bc3fb4f6fc
Submitter: Jenkins
Branch: master

commit 893f30c61d280804e417790dd34ba7bc3fb4f6fc
Author: paul luse <email address hidden>
Date: Wed Aug 12 13:32:50 2015 -0700

EC GET path: require fragments to be of same set

    And if they are not, exhaust the node iter to go get more. The
    problem without this implementation is a simple overwrite where
    a GET follows before the handoff has put the newer obj back on
    the 'alive again' node such that the proxy gets n-1 fragments
    of the newest set and 1 of the older.

    This patch bucketizes the fragments by etag and if it doesn't
    have enough continues to exhaust the node iterator until it
    has a large enough matching set.

    Change-Id: Ib710a133ce1be278365067fd0d6610d80f1f7372
    Co-Authored-By: Clay Gerrard <email address hidden>
    Co-Authored-By: Alistair Coles <email address hidden>
    Closes-Bug: 1457691

Changed in swift:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-02: Fix proposed to swift (feature/crypto)

#17

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/219775

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-03: Fix merged to swift (feature/crypto)

#18

Download full text (43.3 KiB)

Reviewed: https://review.openstack.org/219775
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=257e468e9bfd1088040419ad408106ac3c77b531
Submitter: Jenkins
Branch: feature/crypto

commit e02609c66a804845672413b06830b87395afef31
Author: Samuel Merritt <email address hidden>
Date: Tue Sep 1 15:19:50 2015 -0700

Preserve traceback in swift-dispersion-report

    Commit c690bcb fixed a bug in the dispersion report, but changed this
    from a bare "raise" to "raise err", which loses the traceback. Not a
    big deal, but worth putting back IMO.

Change-Id: Id5b72153a4b8df8e3faaf1fa3fb2040e28ba85cc

commit d06d4ad0fd2dfe69da8008e729651264522c6c06
Author: Minwoo Bae <email address hidden>
Date: Tue Sep 1 15:08:44 2015 -0500

Included reference in swift.obj.diskfile to enumerate the string
used for data file paths.

Change-Id: Ie22caa678bc00dfc43fabec7efbbb9f34490f1b5

commit 524c89b7eeff037b8a6b421888771e15f98c2da2
Author: John Dickinson <email address hidden>
Date: Fri Aug 21 13:39:41 2015 -0700

Updated CHANGELOG, AUTHORS, and .mailmap for 2.4.0 release.

Change-Id: Ic6301146b839c9921bb85c4f4c1e585c9ab66661

commit 05de1305a903ee4ce9c8c50fde53c552d5b90d51
Author: Clay Gerrard <email address hidden>
Date: Thu Aug 27 18:35:09 2015 -0700

Make ssync_sender send valid chunked requests

    The connect method of ssync_sender tells the remote connection that it's
    going to send a valid HTTP chunked request, but if the remote end needs
    to respond with an error of any kind sender throws HTTP right out the
    window, picks up his ball, and closes the socket down hard - much to the
    surprise of the eventlet.wsgi server who up to this point had been
    playing along quite nicely with this 'SSYNC' nonsense assuming that
    everyone here is consenting mature adults.

If you're going to make a "Transfer-Encoding: chunked" request have the
good decency to finish the job with a proper '0\r\n\r\n'. [1]

    N.B. It might be possible to handle an error status during the
    initialize_request phase with some sort of 100-continue support, but
    honestly it's not entirely clear to me when the server isn't going to
    close the connection if the client is still expected to send the body
    [2] - further if the error comes later during missing_check or updates
    we'll for sure want to send the chunk transfer termination line before
    we close down the socket and this way we cover both.

    1. Really, eventlet.wsgi shouldn't be so blasted brittle about this [3]
    2. https://lists.w3.org/Archives/Public/ietf-http-wg/2005AprJun/0007.html
    3. https://github.com/eventlet/eventlet/commit/c3ce3eef0b4d0dfdbfb1ec0186d4bb204fb8ecd5

Closes-Bug #1489587
Change-Id: Ic17c6c3075553f8cf6ef6213e62a00282f0d01cf

commit 993ee4e37af1961adba2047d5aa2eb210e423eb3
Author: nakagawamsa <email address hidden>
Date: Fri Aug 28 11:49:43 2015 +0900

Remove duplicate X-Backend-Storage-Policy-Index key

There is duplicate 'X-Backend-Storage-Policy-Index' dictionary key in unit.obj.test_server.py.
One key has fixed policy index value, and another ha...

Reviewed:  https://review.openstack.org/219775
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=257e468e9bfd1088040419ad408106ac3c77b531
Submitter: Jenkins
Branch:    feature/crypto

commit e02609c66a804845672413b06830b87395afef31
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Sep 1 15:19:50 2015 -0700

Preserve traceback in swift-dispersion-report
    
    Commit c690bcb fixed a bug in the dispersion report, but changed this
    from a bare "raise" to "raise err", which loses the traceback. Not a
    big deal, but worth putting back IMO.
    
    Change-Id: Id5b72153a4b8df8e3faaf1fa3fb2040e28ba85cc

commit d06d4ad0fd2dfe69da8008e729651264522c6c06
Author: Minwoo Bae <minwoob@us.ibm.com>
Date:   Tue Sep 1 15:08:44 2015 -0500

Included reference in swift.obj.diskfile to enumerate the string
    used for data file paths.
    
    Change-Id: Ie22caa678bc00dfc43fabec7efbbb9f34490f1b5

commit 524c89b7eeff037b8a6b421888771e15f98c2da2
Author: John Dickinson <me@not.mn>
Date:   Fri Aug 21 13:39:41 2015 -0700

Updated CHANGELOG, AUTHORS, and .mailmap for 2.4.0 release.
    
    Change-Id: Ic6301146b839c9921bb85c4f4c1e585c9ab66661

commit 05de1305a903ee4ce9c8c50fde53c552d5b90d51
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Aug 27 18:35:09 2015 -0700

Make ssync_sender send valid chunked requests
    
    The connect method of ssync_sender tells the remote connection that it's
    going to send a valid HTTP chunked request, but if the remote end needs
    to respond with an error of any kind sender throws HTTP right out the
    window, picks up his ball, and closes the socket down hard - much to the
    surprise of the eventlet.wsgi server who up to this point had been
    playing along quite nicely with this 'SSYNC' nonsense assuming that
    everyone here is consenting mature adults.
    
    If you're going to make a "Transfer-Encoding: chunked" request have the
    good decency to finish the job with a proper '0\r\n\r\n'. [1]
    
    N.B. It might be possible to handle an error status during the
    initialize_request phase with some sort of 100-continue support, but
    honestly it's not entirely clear to me when the server isn't going to
    close the connection if the client is still expected to send the body
    [2] - further if the error comes later during missing_check or updates
    we'll for sure want to send the chunk transfer termination line before
    we close down the socket and this way we cover both.
    
    1. Really, eventlet.wsgi shouldn't be so blasted brittle about this [3]
    2. https://lists.w3.org/Archives/Public/ietf-http-wg/2005AprJun/0007.html
    3. https://github.com/eventlet/eventlet/commit/c3ce3eef0b4d0dfdbfb1ec0186d4bb204fb8ecd5
    
    Closes-Bug #1489587
    Change-Id: Ic17c6c3075553f8cf6ef6213e62a00282f0d01cf

commit 993ee4e37af1961adba2047d5aa2eb210e423eb3
Author: nakagawamsa <nakagawamsa@nttdata.co.jp>
Date:   Fri Aug 28 11:49:43 2015 +0900

Remove duplicate X-Backend-Storage-Policy-Index key
    
    There is duplicate 'X-Backend-Storage-Policy-Index' dictionary key in unit.obj.test_server.py.
    One key has fixed policy index value, and another has random value.
    Unittest should done with random policy index, so remove key which is set fixed value.
    
    Change-Id: Ic91fcf44d48297d0feee33c928ca682def9790a3

commit 893f30c61d280804e417790dd34ba7bc3fb4f6fc
Author: paul luse <paul.e.luse@intel.com>
Date:   Wed Aug 12 13:32:50 2015 -0700

EC GET path: require fragments to be of same set
    
    And if they are not, exhaust the node iter to go get more.  The
    problem without this implementation is a simple overwrite where
    a GET follows before the handoff has put the newer obj back on
    the 'alive again' node such that the proxy gets n-1 fragments
    of the newest set and 1 of the older.
    
    This patch bucketizes the fragments by etag and if it doesn't
    have enough continues to exhaust the node iterator until it
    has a large enough matching set.
    
    Change-Id: Ib710a133ce1be278365067fd0d6610d80f1f7372
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
    Closes-Bug: 1457691

commit d4409c0a046c6ce0e14e18c95efe2cd16cf120e8
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Aug 11 09:10:13 2015 -0500

Better scoping for tempurls, especially container tempurls
    
    It used to be that a GET of a tempurl referencing a large object would
    let you download that large object regardless of where its segments
    lived. However, this led to some violated user expectations around
    container tempurls.
    
    (Note on shorthand: all tempurls reference objects. However, "account
    tempurl" and "container tempurl" are shorthand meaning tempurls
    generated using a key on the account or container, respectively.)
    
    Let's say an application is given tempurl keys to a particular
    container, and it does all its work therein using those keys. The user
    expects that, if the application is compromised, then the attacker
    only gains access to the "compromised-container". However, with the old
    behavior, the attacker could read data from *any* container like so:
    
    1) Choose a "victim-container" to download
    
    2) Create PUT and GET tempurl for any object name within the
       "compromised-container". The object doesn't need to exist;
       we'll create it.
    
    3) Using the PUT tempurl, upload a DLO manifest with
       "X-Object-Manifest: /victim-container/"
    
    4) Using the GET tempurl, download the object created in step 3. The
       result will be the concatenation of all objects in the
       "victim-container".
    
    Step 3 need not be for all objects in the "victim-container"; for
    example, a value "X-Object-Manifest: /victim-container/abc" would only
    be the concatenation of all objects whose names begin with "abc". By
    probing for object names in this way, individual objects may be found
    and extracted.
    
    A similar bug would exist for manifests referencing other accounts
    except that neither the X-Object-Manifest (DLO) nor the JSON manifest
    document (SLO) have a way of specifying a different account.
    
    This change makes it so that a container tempurl only grants access to
    objects within its container, *including* large-object segments. This
    breaks backward compatibility for container tempurls that may have
    pointed to cross container *LO's, but (a) there are security
    implications, and (b) container tempurls are a relatively new feature.
    
    This works by having the tempurl middleware install an authorization
    callback ('swift.authorize' in the WSGI environment) that limits the
    scope of any requests to the account or container from which the key
    came.
    
    This requires swift.authorize to persist for both the manifest request
    and all segment requests; this is done by having the proxy server
    restore it to the WSGI environment prior to returning from __call__.
    
    [CVE-2015-5223]
    
    Co-Authored-By: Clay Gerrard <clayg@swiftstack.com>
    Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
    Co-Authored-By: Christian Schwede <cschwede@redhat.com>
    Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
    
    Change-Id: Ie6d52f7a07e87f6fec21ed8b0ec1d84be8b2b11c
    Closes-Bug: 1449212

commit 10b2939b433a4a79b4f7b97640b3d208cacfeffb
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Jul 23 22:36:21 2015 -0700

Disallow unsafe tempurl operations to point to unauthorized data
    
    Do not allow PUT tempurls to create pointers to other data. Specifically
    disallow the creation of DLO object manifests by returning an error if a
    non-safe tempurl request includes an X-Object-Manifest header regardless of
    the value of the header.
    
    This prevents discoverability attacks which can use any PUT tempurl to probe
    for private data by creating a DLO object manifest and then using the PUT
    tempurl to head the object which would 404 if the prefix does not match any
    object data or form a valid DLO HEAD response if it does.
    
    This also prevents a tricky and potentially unexpected consequence of PUT
    tempurls which would make it unsafe to allow a user to download objects
    created by tempurl (even if they just created them) because the result of
    reading the object created via tempurl may not be the data which was uploaded.
    
    [CVE-2015-5223]
    
    Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
    
    Change-Id: I11e68830009d3f6bff44ae4011a41b67139146f6
    Closes-Bug: 1453948

commit c690bcb68331818a04c94741b25f40cc40f7b3c4
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Mon Aug 17 16:50:56 2015 +0900

Fix dispersion-reports error message
    
    This patch fixes Swift to show message
    "No objects to query. Has swift-dispersion-populate been run?"
    for "swift-dispersion-report —object-only”
    with no container for object dispersion.
    
    Change-Id: I82da56709cfc296a27f5180681709bc56adbc13d
    Closes-Bug: #1468120

commit 8086a0e53406dc95856df2d3e015d3eaaf81380a
Author: Tushar Gohad <tushar.gohad@intel.com>
Date:   Tue Aug 18 07:24:19 2015 +0000

Restrict PyECLib version to 1.0.7
    
    v1.0.9 rev of PyECLib replaces Jerasure with a native EC
    implementation (liberasurecode_rs_vand) as the default
    EC scheme.  Going forward, Jerasure will not be bundled
    with PyPI version of PyECLib as it used to be, until
    v1.0.7.
    
    This is an interim change to Swift requirements until we
    get v1.0.9 PyECLib included into global-requirements and
    ready patches that change Swift default ec_type (for doc,
    config samples and unit tests) from "jerasure_rs_vand"
    to "liberasurecode_rs_vand."
    
    Change-Id: Ica4fee2cdea2bc7f5edd0e51ad637a4457faf3b4

commit 17efa343c605d0361b3f423696babbab3f3d972d
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Jul 6 13:21:40 2015 -0700

Fix EC GET backend stream iteration state
    
    In EC case, When GET object requested, proxy-server always makes a log
    line "Client disconnected on read" even though the request succeeded.
    
    That is because ECAppIter class doesn't maintain a bunch of backend
    stream when closing the app_iter. It will cause unfortunately
    GeneratorExit on backend stream ResumingGetter.
    
    This patch fixes to set non_client_disconnected to propagate the state
    to the backend streams when the range iteration stopped successful.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    
    Change-Id: I77af9807816bea1444d66534a17e2a210bcf09f8
    Closes-Bug: #1472201

commit 25dc7224b63b039cb3778cca7f9f81f954c6d94e
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Tue Aug 25 11:05:41 2015 +0100

Fix swob.Range docstring
    
    Bad ranges cause a ValueError to be raised, not an empty
    ranges list.
    
    Change-Id: I118bd2f7dc08ff5198870f4093c6eb350506c8ed

commit a7b84f4c51207608adc3a5de7f4f9a629053c0ab
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon Aug 24 23:11:34 2015 -0700

Allow pep8 of a single file
    
    Now you can run
    
       $ tox -e pep8 path/to/file.py [path/to/file2.py [...]]
    
    to run pep8 against just those files[1]. This is quite a bit faster
    than a full pep8 run, and the faster feedback is nice when you're
    fiddling with some formatting to placate pep8.
    
    Of course, you can still run "tox -e pep8" to check the whole source
    tree, just as before this commit.
    
    [1] It'll still run against bin/swift* as well, but that's still a lot
    faster than running against all our .py files.
    
    Change-Id: I81b4363fb95a34ff0f5c346b2b24f2047154f502

commit 6a35d479e8952c854fee6fbef9fe0397a289a6e5
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Tue Aug 25 11:24:49 2015 +1000

Follow up patch to fix a multiline import NITPIC
    
    This change cleans up test/unit/obj/test_replicator.py's imports
    to use only 1 version of multiline import syntaxes (' \' vs '()').
    I don't really mind which, but we should be consistant, at least
    in the same file.
    
    This is a follow up for patch 215857.
    
    Change-Id: Ie2d328c25865b19092c493981a803ee246a9d7a5

commit edde5584affaa983d8db1d294bf1e20a2d4bbb50
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Tue Aug 25 07:52:18 2015 +0900

Fix typo of a comment in replicator
    
    The typo was introduced by patch 138342 (sorry) so I fix it.
    
    Change-Id: Id5126802d281ef7ee9be128bd2152c0d2584160e

commit 6151c6c45da481c0ea9a4fb876d14b39c1e347bf
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Aug 24 20:36:22 2015 +0000

Fix use of delimiter in account listings
    
    Previously, account listings that used the delimiter query param could
    omit some containers if they ended with the character that follows the
    delimiter.
    
    See If196e3075612b121ef8da4a9128167d00a248c27 for the corresponding fix
    for container listings.
    
    Change-Id: I57fcb97e51f653f5f4e306a632fcb3a0fb148c4e

commit a38f63e1c6b8b85b1675aa900e239a2e9906811e
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Aug 21 18:14:55 2015 -0700

Use correct Storage-Policy header for REPLICATE requests
    
    Under some concurrency the object-replicator could potentially send the
    wrong X-Backed-Storage-Policy-Index header to it's partner nodes during
    replication if there were multiple storage policies on the same node
    because of a race where multiple jobs being processed concurrently would
    mutate some shared state on the ObjectReplicator instance.
    
    Instead of using shared stated on the ObjectReplicator instance when
    mutating the default headers send with REPLICATION requests each job
    will copy them into a local where they can safely be updated.
    
    Change-Id: I5522db57af7e308b1f9d4181f14ea14e386a71fd

commit ae129bf5e8c918efd8ee66279f98ee85e7c97b24
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Fri May 15 06:44:12 2015 -0700

eventlet.posthook related code needn't anymore
    
    In the past, older Swift seemed to use eventlet.posthooks scheme for the
    logging on proxy with posthooklogger method. However, for now, Swift
    attaches no method to eventlet.posthooks everywhere so we don't have
    to maintain the posthooks anymore.
    
    Change-Id: Ie63941a202d448532d980252a4d25575f8edab9c

commit 4500ff340f021e4e5f6fd53960de03cb3c207925
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed May 13 00:43:59 2015 -0700

Fix the missing SLO state on fast-post
    
    When using fast-post and POST (i.e. metadata update) is requested to
    a SLO manifest files, current Swift drops the 'X-Static-Large-Object'
    header from the existing metadata. It results in breaking the SLO
    state because the manifest missing the 'X-Static-Large-Object' metadata
    will be maintained as a normal files.
    
    This patch fixes object-server to keep the existing
    'X-Static-Large-Object' flag and then keep the SLO state.
    
    Change-Id: Ib1eb569071372c322dd105c52baeeb094003291e
    Closes-bug: #1453807

commit 69e7424d3cb6c59341d739c3d41f42397031d96a
Author: Eran Rom <eranr@il.ibm.com>
Date:   Tue Jul 28 09:41:12 2015 +0300

Add container sync probe test to SAIO default set
    
    SAIO Configuration and documentation changes enabling to run the
    container sync probe test by default
    
    Change-Id: Iccf59533d0d4fe72549d318339ab125d04dde006
    Related-Bug: #1476623

commit 8b1df9918bf6c22b49b161a402199b532bfd8266
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Aug 21 14:04:46 2015 -0700

Minor cleanup handoff mode warnings
    
      * message is a little clearer
      * test is a little stronger
    
    Change-Id: I745cde7f4a46dafc80ab42d39e6ccc92aa3b746e

commit a1ceab5a92dfdafd5abffed272ac2cece5302f32
Author: Carlos Cavanna <ccavanna@ca.ibm.com>
Date:   Fri Aug 21 14:14:31 2015 -0400

New troubleshooting case in documentation.
    
    Added a new troubleshooting case for the "First Contribution to Swift"
    documentation page.
    
    Change-Id: I182ba702b49b28409fe56becae93326e5f63dcd0

commit ab163702de733be39ba5e7024c7a8dd4c86bc29b
Author: Pradeep Kumar Singh <pradeep.singh@nectechnologies.in>
Date:   Tue Jul 14 10:03:18 2015 +0530

Emit warning log in object replicator
    
    When the object-replicator encounters handoffs_first and
    handoff_delete options as enabled it should emit a log
    warning indicating that it should be changed back to the
    default before the next "normal" rebalance.
    
    Closes-Bug: #1457262
    
    Change-Id: If9dc2796c18ed3cf13da920831e2d5c2ae9f12a0

commit be66aa8e76d2994eb8a0a944e0dea26ffbb3c159
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Aug 20 13:24:38 2015 -0700

Fix 500 for bogus Range request to 0-byte object.
    
    The proxy was trying to pop a byterange off a Range header that didn't
    contain syntactically-valid byteranges. This worked about as well as
    you'd expect. Now we detect the bogus value and remove the header
    entirely.
    
    Change-Id: I24b92f900d33ec79880c7db2870378489d5a6810

commit 4b6836b3bc465e9c2c6d2974c193a49b973c4e4e
Author: Akihito Takai <takaiak@nttdata.co.jp>
Date:   Thu Aug 20 18:40:52 2015 +0900

Fix the comment in [1].
    
    (line 259) parameter(op) of object_update method is 'PUT' or
    'DELETE' not 'POST' or 'DELETE'.
    
    [1]: swift/obj/updater.py
    
    Change-Id: I876a620ba8e09e69fba7156b12e69445c229e160

commit 923238aa1ba0963c414ba5321cd3910b2910f4ed
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Tue Jul 28 20:35:25 2015 +0530

test/(functional/probe):Replace python print operator with print function (pep H233, py33)
    
    'print' function is compatible with 2.x and 3.x python versions
    Link : https://www.python.org/dev/peps/pep-3105/
    
    Python 2.6 has a __future__ import that removes print as language syntax,
    letting you use the functional form instead
    
    Change-Id: I416c6ac21ccbfb91ec328ffb1ed21e492ef52d58

commit 47dc31940d64b0c87e99cc73eddcdd01dd8b3ad4
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Aug 19 10:42:28 2015 -0700

Add OpenStack release names to changelog
    
    I can never remember the mapping of real version numbers to the
    OpenStack names.
    
    Change-Id: Ib7c5ae4ff2a33018364698eb34f8df1622719fab

commit 183508710d011ec30ebba2186c58fd28d449e4bb
Author: Victor Stinner <vstinner@redhat.com>
Date:   Sun Aug 16 11:18:49 2015 +0200

pep8: Don't override '_' symbol
    
    Fix pep8 warning F402 "import '_' from line 51 shadowed by loop
    variable".
    
    Change-Id: I139060ff5d298a8b0f0f8e529a5737478fb5daf5

commit c7eb589c6ca2fe4f2950cae22a18513e4dec6c46
Author: Victor Stinner <vstinner@redhat.com>
Date:   Sun Aug 16 11:06:52 2015 +0200

pep8: Fix usage of the l10n _('...') function
    
    Fix the pep8 warning H702 "Formatting operation should be outside of
    localization method call".
    
    For the logger, pass parameters as indexed parameters instead of using
    the string str%args operator, the logger is more reliable in case of
    formatting error.
    
    Change-Id: If418bc155f6a6c0a00f63e3d87ebe4addf4aae55

commit 7bea148d2fe22daa9d1fbbc5cd16ebd390b64b1b
Author: Victor Stinner <vstinner@redhat.com>
Date:   Sun Aug 16 10:34:26 2015 +0200

pep8: replace deprecated calls to assert_()
    
    The TestCase.assert_() has been deprecated in Python 2.7. Replace it
    with assertTrue() or even better methods (assertIn, assertNotIn,
    assertIsInstance) which provide better error messages.
    
    Change-Id: I21c730351470031a2dabe5238693095eabdb8964

commit 8aaacbf88d9ea4432e58dd261b98ff95d7c69be6
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Jul 27 18:55:01 2015 +0200

pep8: Fix hacking H232 warnings (octal)
    
    Fix warnings "H232: Python 3.x incompatible octal 000001234 should be
    written as 0o1234".
    
    Change-Id: I9a7bbb034357783885ac3e18fe1e9e32a5951616

commit d6267c1f95a7fcd2b0e4f15d5efc6bcfe2114de0
Author: Clément Contini <ccontini@cloudops.com>
Date:   Wed Aug 12 15:00:45 2015 -0400

Keep user id and project id in subrequests env
    
    Keep HTTP_X_USER_ID and HTTP_X_PROJECT_ID to be available as
    user_id and project_id in storage.objects.outgoing.bytes in
    ceilometer when downloading a multipart object.
    
    Change-Id: I0f4734f021e5d6e84d48ed9bebeb321d7a9590ad
    Closes-Bug: #1477283

commit eaa006464cb98a2f36809edfa13bd3dcaebc9952
Author: John Dickinson <me@not.mn>
Date:   Mon Aug 17 22:13:42 2015 -0700

move global statement up a few lines
    
    Change-Id: I190d2d530c6c0525d988cc88b0965b48a291fffb

commit 1a81cda8b7eba0d25ea7341da756bb588cae8d73
Author: Hamdi Roumani <roumani@ca.ibm.com>
Date:   Mon Aug 3 17:45:56 2015 -0400

Doc instructions for post rebase steps
    
    Improve the 'first_contribution_swift' by adding instructions for how to
    rebuild swift following a rebase.
    
    Change-Id: If5c91dc4e1e8d1712bbd8b326c675967fb4b8c15

commit 79ba4a85983641e539b620bd143e62673c98416e
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Wed Dec 3 06:15:16 2014 +0900

Enable Object Replicator's failure count in recon
    
    This patch makes the count of object replication failure in recon.
    And "failure_nodes" is added to Account Replicator and
    Container Replicator.
    
    Recon shows the count of object repliction failure as follows:
    $ curl http://<ip>:<port>/recon/replication/object
    {
        "replication_last": 1416334368.60865,
        "replication_stats": {
            "attempted": 13346,
            "failure": 870,
    	"failure_nodes": {
                "192.168.0.1": {"sdb1": 3},
                "192.168.0.2": {"sdb1": 851,
                                "sdc1": 1,
                                "sdd1": 8},
                "192.168.0.3": {"sdb1": 3,
                                "sdc1": 4}
    	},
            "hashmatch": 0,
            "remove": 0,
            "rsync": 0,
            "start": 1416354240.9761429,
            "success": 1908
        },
        "replication_time": 2316.5563162644703,
        "object_replication_last": 1416334368.60865,
        "object_replication_time": 2316.5563162644703
    }
    
    Note that 'object_replication_last' and 'object_replication_time' are
    considered to be transitional and will be removed in the subsequent
    releases. Use 'replication_last' and 'replication_time' instead.
    
    Additionaly this patch adds the count in swift-recon and it will be
    showed as follows:
    $ swift-recon object -r
    ========================================================================
    =======
    --> Starting reconnaissance on 4 hosts
    ========================================================================
    =======
    [2014-11-27 16:14:09] Checking on replication
    [replication_failure] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%,
    no_result: 0, reported: 4
    [replication_success] low: 3, high: 3, avg: 3.0, total: 12,
    Failed: 0.0%, no_result: 0, reported: 4
    [replication_time] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%,
    no_result: 0, reported: 4
    [replication_attempted] low: 1, high: 1, avg: 1.0, total: 4,
    Failed: 0.0%, no_result: 0, reported: 4
    Oldest completion was 2014-11-27 16:09:45 (4 minutes ago) by
    192.168.0.4:6002.
    Most recent completion was 2014-11-27 16:14:19 (-10 seconds ago) by
    192.168.0.1:6002.
    ========================================================================
    =======
    
    In case there is a cluster which has servers, a server runs with this
    patch and the other servers run without this patch. If swift-recon
    executes on the server which runs with this patch, there are unnecessary
    information on the output such as [failure], [success] and [attempted].
    Because other servers which run without this patch are not able to
    send a response with information that this patch needs.
    Therefore once you apply this patch, you also apply this patch to other
    servers before you execute swift-recon.
    
    DocImpact
    Change-Id: Iecd33655ae2568482833131f422679996c374d78
    Co-Authored-By: Kenichiro Matsuda <matsuda_kenichi@jp.fujitsu.com>
    Co-Authored-By: Brian Cline <bcline@softlayer.com>
    Implements: blueprint enable-object-replication-failure-in-recon

commit eeb0fa40a19917e6548f95f0bd3c08736928449b
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Aug 6 07:21:15 2015 +0000

Make swift-ring-builder filename usage more consistent
    
    Sometimes the given argument is internally altered and another filename is used
    without a note to the operator. Even worse, a given .ring.gz filename is
    sometimes written out as builder file, without updating the corresponding
    .builder file.
    
    There is already a method to parse the given argv and return the name of the
    builder and ring file. However, it's rarely used and no warning is given to the
    user if it is altered. This patch uses the already parsed builder and ring file
    name instead of argv[1], and also adds a note to the user if the used filename
    is differently to the one given as argument.
    
    Closes-Bug: 1482096
    Change-Id: I2f8ef23aeab8b07caaa799f7dcd57e684b4b2ad2

commit b75d2a4e37d4c86763a2cc56c6dd53ebe2e0de19
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Mon Aug 17 13:54:44 2015 -0500

Quorum on durable response is too low
    
    Increase the .durable quorum from 2 to "parity + 1" to guarantee
    that we will never fail to rebuild an object.  Otherwise, with
    low durable responses back (< parity + 1), the putter objects
    return with failed attribute set to true, thereby failing the
    rebuild of fragments for an object.
    
    Change-Id: I80d666f61273e589d0990baa78fd657b3470785d
    Closes-Bug: 1484565

commit fa35e38c9f83f704eb087f80157cbe33b23b9db2
Author: Kai Zhang <zakir.exe@gmail.com>
Date:   Fri Aug 14 16:46:35 2015 -0700

Fix some minor typos
    
    Fixed some typos in function name and comments.
    
    Change-Id: Ida76ab4b331a51b71e57650702acc136e66ba4b2

commit 5b246e875fe9ac3d764ea581ad52b04238f5bcc8
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Jul 22 10:39:22 2015 -0700

Fix EC range GET/COPY handling
    
    When range GET (or COPY) for an EC object requested, if the requested range
    starts from more than last segments alignment (i.e.
    ceil(object_size/segment_size) * segment_size), proxy server will return
    the original content length w/o body, though Swift should return an error
    massage as a body and the length of message as the content length.
    The current behavior will cause stuck on some client. (e.g. curl)
    
    This patch fixes that proxy enables to return correct response, even if such
    an over range requested.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    
    Change-Id: I21f81c842f563ac4dddc69011ed759b744bb20bd
    Closes-Bug: #1475499

commit 7064706b2721c36c5181267f58704b9344ad4ac0
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Aug 11 10:17:20 2015 -0500

Fix 500 in versioned writes with bad Destination
    
    When this code lived in the proxy, it was protected by an "except
    HTTPException" clause in proxy.Application.handle_request(). When it
    moved to middleware, it lost that, and then things like
    constraints.check_name_format that raised HTTPException would cause
    500s. The HTTPException would make it all the way out to catch_errors
    and get translated to a 500.
    
    This commit just wraps a couple try/excepts around the bits in
    versioned writes that can raise HTTPException. I tried to make it use
    wsgify so I could get that for free, but that wound up being a real
    pain because env/start_response are plumbed through pretty much the
    whole versioned-writes middleware.
    
    Closes-Bug: 1483705
    
    Change-Id: Ife165bf709e64f313ed07c779b21914045e51f25

commit 035a411660ca02983cd486312266c67d78a2359c
Author: Thiago da Silva <thiago@redhat.com>
Date:   Sun Nov 9 13:13:27 2014 -0500

versioned writes middleware
    
    Rewrite object versioning as middleware to simplify the PUT method
    in the object controller.
    
    The functionality remains basically the
    same with the only major difference being the ability to now
    version slo manifest files. dlo manifests are still not
    supported as part of this patch.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    
    DocImpact
    Change-Id: Ie899290b3312e201979eafefb253d1a60b65b837
    Signed-off-by: Thiago da Silva <thiago@redhat.com>
    Signed-off-by: Prashanth Pai <ppai@redhat.com>

commit 968c91a465627bb9317ed6e2ae9ccd818c83c0c4
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Tue Jul 28 10:31:54 2015 +0100

Replace assertTrue(not <condition>) with assertFalse(<condition>)
    
    The replacement of assert_ with assertTrue [1] resulted in a number of
    tests using calls of the form assertTrue(not <condition>). This patch
    replaces those with assertFalse(<condition>).
    
    [1] change I74705c6498249337bfdf955d62e0ad972035bc1f
    
    Change-Id: I78b49558f4425c2335df187b1793d1e4b3c514b1

commit 4ac1fea5d111c669ff827f4eb29c0735cbad6ba5
Author: Zhao Lei <zhaolei@cn.fujitsu.com>
Date:   Fri Aug 7 22:07:01 2015 +0800

Fix some spelling typo in comments
    
    s/overide/override for object-expirer.conf and sample.
    s/automaticaly/automatically for swift/proxy/controllers/obj.py
    
    Change-Id: Ife107c7a1005a5d4959288db50a7f8f33c522dd4
    Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>

commit 9456af35a2832aeafc4a4e78ebd0d4142ead71cd
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Thu Aug 6 00:55:36 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3,changes in
    dir:
    *test/unit/obj/*
    *test/unit/test_locale/*
    
    Change-Id: I3dd0c1107165ac529f1cd967363e5cf408a1d02b

commit 1952451ed7e97e2bfa3b11dfa5000cc39f712084
Author: Zhao Lei <zhaolei@cn.fujitsu.com>
Date:   Fri Aug 7 21:57:08 2015 +0800

Fix a spelling typo in comment
    
    s/automaticaly/automatically/ for swift/proxy/controllers/obj.py
    
    Change-Id: I405441383739637e0b13746d6f4e2bad1c874b4d
    Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>

commit c35cc13b8abeb97400632cbe8ec56fc1040d8210
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Thu Aug 6 10:01:17 2015 -0500

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in the following dir:
    test/unit/proxy/*
    
    Change-Id: Ie2c7e73e1096233a10ee7fbf6f88386fa4d469d6

commit 239e94e62518f36601fb18119700269753d02a01
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Thu Aug 6 09:28:51 2015 -0500

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in the following dir:
    test/probe/*
    
    Change-Id: Ie08dd7a8a6c48e3452dfe4f2b41676330ce455d5

commit 49b9ba37ac522502ccad66fba7bfef375eab145d
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Wed Aug 5 14:51:32 2015 -0500

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in the following dir:
    test/functional/*
    
    Change-Id: Iee7f8ffca9838ccc521107180697d91ac9559405

commit 81816bebe6b3506af8cabaa7ebe265c70f3499cd
Author: kenichiro matsuda <matsuda_kenichi@jp.fujitsu.com>
Date:   Thu Aug 6 10:13:02 2015 +0900

Fix shebang of commands
    
    Fix shebang of following commands.
      $ grep '#!/usr/bin/python' swift/bin/*
      swift/bin/swift-account-info:#!/usr/bin/python
      swift/bin/swift-container-info:#!/usr/bin/python
      swift/bin/swift-container-sync:#!/usr/bin/python
      swift/bin/swift-recon:#!/usr/bin/python
      swift/bin/swift-ring-builder:#!/usr/bin/python
      swift/bin/swift-ring-builder-analyzer:#!/usr/bin/python
    
    Change-Id: I564d1d8abd76eba57730fc2f30263b0a0f809867
    Closes-Bug: #1481623

commit 652f0f9da408e955843f87a504b3dab5be3cff80
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Wed Aug 5 15:33:29 2015 -0600

Having said H, I, J, we ought to say K
    
    In the long run, we might want to stop these updates. But this
    decision is above my pay grade.
    
    Change-Id: I335558e1da8052be1b215fbad51244b47af3d81b

commit ed3aec2146b114c85bb3b196c999bf3f44fe159e
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Thu Aug 6 00:18:52 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in
    dir: test/unit/container/*
    
    Change-Id: I3333022ed63ce03198bc73147246d91d2442a440

commit f449e914728fdcaf0870c15b9fc5ebd16a9df916
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Wed Aug 5 22:32:02 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3, fixing in:
    dir: test/unit/cli/*
    
    Change-Id: I9a2fc1f717beafd5fa8408942046e310e8de0318

commit 6594bbebb582801c069c4966352fee19eadef2f8
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Wed Aug 5 22:25:46 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in
    dir : test/unit/account/*
    
    Change-Id: I70415197df1b0d58d4d6e219733c13a9c9cdfff7

commit 89f59062864e5cbfc839a6084c323ce35438aa57
Author: Ben Martin <blmartin@us.ibm.com>
Date:   Mon Jul 27 14:19:09 2015 -0500

+Document method to avoid rsync filling root drive
    
    When rsync pushes to a remote node with an unmounted drive and if
    certain steps are not taken, rsync may attempt to write files to
    the local drive at the location where the drive was mounted.
    
    There are two suggested solutions for this issue:
      1) Set the permissions for all mount points in /srv/node/
           to root:root 755
      2) Mount the drives elsewhere and symlink the drives to /srv/.../
    
    The first method ensures that only root and not the swift user
    can write in the /srv/.../ directories.
    
    The second method will prompt a broken link issue if rsync
    attempts to write to an unmounted drive.
    
    Change-Id: I60ce4ed9ef8401768d5f78b6806cbb2e2a65303e
    Closes-Bug: #1470576

commit f0d51882b921075f5309ed6ff07385dc6103d4dc
Author: Charles Hsu <charles0126@gmail.com>
Date:   Wed Aug 5 22:09:40 2015 +0800

Add extra_header_count to document and config.
    
    Change-Id: Iec86b488d71553c295afe7098822ce2046df9546

commit 7071762d3698c59fabdee76890e300f81417be74
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Aug 4 23:15:37 2015 -0700

Fix TypeError if backend response doesn't have expected headers
    
    There was some debug logging mixed in with some error handling on PUTs
    that was relying on a very specific edge would only encounter a set of
    backend responses that included the expected set of headers to diagnoise
    the failure.
    
    But the backend responses may not always have the expected headers.
    
    The proxy debug logging should be more robust to missing headers.
    
    It's a little hard to follow, but if you look `_connect_put_node` in
    swift.proxy.controller.obj - you'll see that only a few connections can
    make their way out of the initial put connection handling with a "resp"
    attribute that is not None.  In the happy path (e.g. 100-Continue) it's
    explictly set to None, and in most errors (Timeout, 503, 413, etc) a new
    connection will be established to the next node in the node iter.
    
    Some status code will however allow a conn to be returned for validation
    in `_check_failure_put_connections`, i.e.
    
      * 2XX (e.g. 0-byte PUT would not send Expect 100-Continue)
      * 409 - Conflict with another timestamp
      * 412 - If-None-Match that encounters another object
    
    ... so I added tests for those - fixing a TypeError along the way.
    
    Change-Id: Ibdad5a90fa14ce62d081e6aaf40aacfca31b94d2

commit 01848c565e5821d2500cb234261c70871e562ebe
Author: David Goetz <dpgoetz@gmail.com>
Date:   Fri Jun 26 17:27:31 2015 -0700

Speed up reaper for a big account delete and some better error handling
    
    In run_forever mode, run the reaper on all three copies of the account
    DB instead of just the first one and shard the containers between them.
    
    Change-Id: Id182e89dad70df9d6d9988523861b8085d74c765

commit 5b24b2249878e25ee702ee4237227c984036c6ec
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Tue Aug 4 06:29:11 2015 +0000

Imported Translations from Transifex
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I9f0e5fd25ec143fb87de1895794764dea4587f93

commit 8d06f56c1fac23bca463d3a93353da6b50ad5bed
Author: Takashi NATSUME <natsume.takashi@lab.ntt.co.jp>
Date:   Tue Aug 4 10:14:25 2015 +0900

Fix typo of 'receive'
    
    Change-Id: Ic01defa8d48ba4b7f4bd6cbe4d59b9bab468a36c

commit 310ac18d61b001bdf297505243d8aa62e8fde8fe
Author: Brian Reitz <brian.reitz@oracle.com>
Date:   Mon Aug 3 16:28:31 2015 -0500

Convert docstring to a comment in test_denied_DELETE_of_versioned_object unit test.
    
    Docstrings cause issues for nose and created non standard test output for this test.
    
    Change-Id: I91e6450480567de0ee0b18d3efae01a4b94f0a86

commit 21a7b4aaa6f991c0eeb8f74876b14fe22713586b
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Mon Aug 3 14:58:24 2015 +0100

Test that get_hashes ignores only removed hash dir
    
    Add test for case not yet covered by unit tests: suffix dir
    has two hash dirs, one with expired tombstone. That hash dir gets
    removed and its hash is not included in the suffix hash, but the
    remaining hash dir's hash is reported.
    
    Change-Id: I031a022daed6b8a66dfd04bea1b4d5eebcb882b3

commit 44917c8a90e79a85cd6c6821d8b6d043f83b322d
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jul 20 13:06:48 2015 -0700

Handle removed suffix dirs the same as empty suffix dirs
    
    When hashes suffix dirs, the directory might have gotten cleaned up while
    still appearing in hashes.pkl.  It would even get cleaned up the next time.
    
    For example, given this really old tombstone:
    
        objects/846/3d0/d3a20154d0a828a032aba6860397c3d0/1432267961.41808.ts
    
    Prior to this commit, a call to get_hashes() would reap the old
    tombstone and any empty containing dirs, but the resulting hashes.pkl
    would still contain {'3d0': 'd41d8cd98f00b204e9800998ecf8427e'} even
    though there's no such suffix dir any more.
    ("d41d8cd98f00b204e9800998ecf8427e" is the MD5 of the empty string.)
    
    Then, a *subsequent* get_hashes() call would omit 3d0 from the resulting
    hash, so then hashes.pkl would no longer contain 3d0. This difference
    would result in a little useless replication traffic while nodes without
    a particular part/suffix pair, but who disagreed on how that showed up
    in hashes.pkl, tried to push their version of nothing to one another.
    
    Now, an empty suffix dir doesn't appear in hashes.pkl at all, whether
    it's for replication or EC, or whether it's for the get_hashes() call
    that reaped the suffix dirs or not.
    
    Co-Author: Samuel Merritt <sam@swiftstack.com>
    
    Change-Id: Ie1bfb1cc56d0fc030c6db42f97b55d140695cf1f

commit cf4d50bd68b658d35759fb56446557858dcd9572
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Fri Jun 5 13:06:29 2015 +0100

Make test_proxy work independent of env vars
    
    test.unit.common.test_internal_client.TestSimpleClient.test_proxy
    will fail if the environment has http_proxy set and does not have
    no_proxy=127.0.0.1 set. This seems to be because urllib overrides
    any proxy arg passed to the Request with the env http_proxy var,
    unless the Request host is in no_proxy.
    
    We don't need to test urllib, so this patch changes the test to
    simply ensure that swift code does pass the correct proxy arg to
    urllib2.urlopen. That avoids testers having to make sure their
    env vars are compliant to successfully run unit tests. While it is
    reasonable to think that 127.0.0.1 is in the no_proxy list, it
    shouldn't be a requirement pass swift tests.
    
    Change-Id: Iff91acdb76fabe7056b3e90e2756e27fe5105817

commit 67513fc17c23066f6547b5e3c4ab2dc852e4f7f0
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sat Jun 27 23:26:47 2015 +0530

Adding bandit for security static analysis testing in swift
    
    This change adds a basic bandit config for Swift. It can be invoked
    by running the tox environment for bandit;
        tox -e bandit
    
    This is an initial step for using bandit with Swift
    and it should be revisited to improve the testing as more is learned
    about the specific needs of the Swift code base.As per now some tests
    are excluded as they were used on purpose.
    
    https://wiki.openstack.org/wiki/Security/Projects/Bandit
    
    Implements: blueprint swift-bandit
    
    Change-Id: I621be9a68ae9311f3a6eadd1636b05e646260cf2

commit 768d7ab074807175471d589de96d091e3239052e
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Jul 30 15:30:35 2015 -0700

Add a probetest for HUP/reload
    
    This would have been enough to catch the regression, and we can extend
    them as we work on any future ehancements to our process management.
    
    Change-Id: I9a1b57aa15663380c45cf783afc8212ab4ffbace

commit c5b5cf91a984f80cc6cbe42735b242083d700542
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Tue Jul 28 21:03:05 2015 +0530

test/unit: Replace python print operator with print function (pep H233, py33)
    
    'print' function is compatible with 2.x and 3.x python versions
    Link : https://www.python.org/dev/peps/pep-3105/
    
    Python 2.6 has a __future__ import that removes print as language syntax,
    letting you use the functional form instead
    
    Change-Id: I94e1bc6bd83ad6b05695c7ebdf7cbfd8f6d9f9af

commit dd2f1be3b124d3901ebbc176a7adc462b6449667
Author: Ondrej Novy <ondrej.novy@firma.seznam.cz>
Date:   Tue Jun 16 17:42:58 2015 +0200

Time synchronization check in recon.
    
    This change add call time to recon middleware and param --time to
    recon CLI. This is usefull for checking if time in cluster is
    synchronized.
    
    Change-Id: I62373e681f64d0bd71f4aeb287953dd3b2ea5662

commit cbcfb74132aba4d5834dc9cf33dc4f664b82d39a
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed Jul 22 19:32:46 2015 -0700

Cleanup nits from container_update_timeout
    
    Change-Id: Ib17fc9a027db267350abc5a7db4e3172dfff7913

commit 2289137164231d7872731c2cf3d81b86f34f01a4
Author: John Dickinson <me@not.mn>
Date:   Sat May 23 15:40:03 2015 -0700

do container listing updates in another (green)thread
    
    The actual server-side changes are simple. The tests are a different
    matter. Many changes were needed to the object server tests to
    handle the now-async calls to the container server. In an effort to
    test this properly, some drive-by changes were made to improve tests.
    
    I tested this patch by doing zero-byte object writes to one container
    as fast as possible. Then I did it again while also saturating 2 of the
    container replica's disks. The results are linked below.
    
    https://gist.github.com/notmyname/2bb85acfd8fbc7fc312a
    
    DocImpact
    
    Change-Id: I737bd0af3f124a4ce3e0862a155e97c1f0ac3e52

commit c8131c6abb17b00b043a9b95e65be819e1fd3c9d
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sun Jul 12 14:48:35 2015 +0530

sys.exc_type/exc_value/exc_traceback are Deprecated
    
    sys.exc_info() contains a tuple of these three.
    
    Change-Id: I530cbeb37c43da98b4924db41f6604871077bd47

tags:

added: in-feature-crypto

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-08: Fix proposed to swift (feature/hummingbird)

#19

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/221410

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-08: Fix merged to swift (feature/hummingbird)

#20

Download full text (70.7 KiB)

Reviewed: https://review.openstack.org/221410
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=eb8f1f83f1cfc63d8452bc30096fd1c145781527
Submitter: Jenkins
Branch: feature/hummingbird

commit cb683d391cb66d0f52830de16760c80fd2afedf9
Author: OpenStack Proposal Bot <email address hidden>
Date: Sat Sep 5 06:17:51 2015 +0000

Imported Translations from Transifex

For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I2d92b8e34a665fb0bb4c048cfb0c59de295dfce6

commit e4542455c8a07b7981c247df8b737816062c1655
Author: Emett Speer <email address hidden>
Date: Wed Sep 2 17:18:03 2015 -0700

[Labs] Update links to Cloud Admin Guide

Update links to the Cloud Admin Guide after the
RST conversion of that book altered URLs.

Change-Id: I899f8938498b744e62887968a65e58c00ef27f1b

commit 58fcc07523978306cd3889ada73af5d9e664cf59
Author: Christian Schwede <email address hidden>
Date: Wed Sep 2 10:52:34 2015 +0000

Test if container_sweep is executed on unmounted devices

This change ensures that container_sweep is not run if a device is not mounted
and mount_check is set to True.

Change-Id: I823083c8431d9e61fd426508033ec9188503957b

commit e02609c66a804845672413b06830b87395afef31
Author: Samuel Merritt <email address hidden>
Date: Tue Sep 1 15:19:50 2015 -0700

Preserve traceback in swift-dispersion-report

    Commit c690bcb fixed a bug in the dispersion report, but changed this
    from a bare "raise" to "raise err", which loses the traceback. Not a
    big deal, but worth putting back IMO.

Change-Id: Id5b72153a4b8df8e3faaf1fa3fb2040e28ba85cc

commit d06d4ad0fd2dfe69da8008e729651264522c6c06
Author: Minwoo Bae <email address hidden>
Date: Tue Sep 1 15:08:44 2015 -0500

Included reference in swift.obj.diskfile to enumerate the string
used for data file paths.

Change-Id: Ie22caa678bc00dfc43fabec7efbbb9f34490f1b5

commit 615c7a204b9386e05c5bab658bfe96766ad1e680
Author: Brian Cline <email address hidden>
Date: Tue Sep 1 10:51:20 2015 -0500

Adds useful dispersion info from changelog

Change-Id: I1a45088fc32620b02ff9a754b02ec1eb75a59d6e

commit 3b8755098a1786c5447abf158bd686293a82977c
Author: janonymous <email address hidden>
Date: Sun Aug 2 21:29:13 2015 +0530

Replace a / b with a // b to use integer division where needed

Change-Id: I72c81faa62786e140b0de00e3a04934bf1b5adbd

commit 524c89b7eeff037b8a6b421888771e15f98c2da2
Author: John Dickinson <email address hidden>
Date: Fri Aug 21 13:39:41 2015 -0700

Updated CHANGELOG, AUTHORS, and .mailmap for 2.4.0 release.

Change-Id: Ic6301146b839c9921bb85c4f4c1e585c9ab66661

commit 05de1305a903ee4ce9c8c50fde53c552d5b90d51
Author: Clay Gerrard <email address hidden>
Date: Thu Aug 27 18:35:09 2015 -0700

Make ssync_sender send valid chunked requests

    The connect method of ssync_sender tells the remote connection that it's
    going to send a valid HTTP chunked request, but if the remote end needs
    to respond with an error of any kind sender th...

Reviewed:  https://review.openstack.org/221410
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=eb8f1f83f1cfc63d8452bc30096fd1c145781527
Submitter: Jenkins
Branch:    feature/hummingbird

commit cb683d391cb66d0f52830de16760c80fd2afedf9
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Sat Sep 5 06:17:51 2015 +0000

Imported Translations from Transifex
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I2d92b8e34a665fb0bb4c048cfb0c59de295dfce6

commit e4542455c8a07b7981c247df8b737816062c1655
Author: Emett Speer <speer.emett@gmail.com>
Date:   Wed Sep 2 17:18:03 2015 -0700

[Labs] Update links to Cloud Admin Guide
    
    Update links to the Cloud Admin Guide after the
    RST conversion of that book altered URLs.
    
    Change-Id: I899f8938498b744e62887968a65e58c00ef27f1b

commit 58fcc07523978306cd3889ada73af5d9e664cf59
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed Sep 2 10:52:34 2015 +0000

Test if container_sweep is executed on unmounted devices
    
    This change ensures that container_sweep is not run if a device is not mounted
    and mount_check is set to True.
    
    Change-Id: I823083c8431d9e61fd426508033ec9188503957b

commit e02609c66a804845672413b06830b87395afef31
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Sep 1 15:19:50 2015 -0700

Preserve traceback in swift-dispersion-report
    
    Commit c690bcb fixed a bug in the dispersion report, but changed this
    from a bare "raise" to "raise err", which loses the traceback. Not a
    big deal, but worth putting back IMO.
    
    Change-Id: Id5b72153a4b8df8e3faaf1fa3fb2040e28ba85cc

commit d06d4ad0fd2dfe69da8008e729651264522c6c06
Author: Minwoo Bae <minwoob@us.ibm.com>
Date:   Tue Sep 1 15:08:44 2015 -0500

Included reference in swift.obj.diskfile to enumerate the string
    used for data file paths.
    
    Change-Id: Ie22caa678bc00dfc43fabec7efbbb9f34490f1b5

commit 615c7a204b9386e05c5bab658bfe96766ad1e680
Author: Brian Cline <bcline@softlayer.com>
Date:   Tue Sep 1 10:51:20 2015 -0500

Adds useful dispersion info from changelog
    
    Change-Id: I1a45088fc32620b02ff9a754b02ec1eb75a59d6e

commit 3b8755098a1786c5447abf158bd686293a82977c
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sun Aug 2 21:29:13 2015 +0530

Replace a / b with a // b to use integer division where needed
    
    Change-Id: I72c81faa62786e140b0de00e3a04934bf1b5adbd