Bug #1364752 “container reconciler write 499 and 404 status code...” : Bugs : OpenStack Object Storage (swift)

Anton (hettbox) on 2014-09-03

summary:

- container reconciler write 499 and 404 error
+ container reconciler write 499 and 404 status code

Revision history for this message

Gil Vernik (gilv) wrote on 2014-09-03:

#1

I wonder if this somehow related this one https://bugs.launchpad.net/swift/+bug/1349688

Revision history for this message

clayg (clay-gerrard) wrote on 2014-09-03:

#2

consume app_iter before break Edit (511 bytes, text/plain)

The 404 is expected, the reconciler makes some optimistic requests for containers that may have enqueued misplaced objects, if there are no such objects the 404's get logged - which seems fine, is normal and working as expected.

The 499's where a surprise (to me at least) - I either hadn't noticed them before or I wasn't expecting to see the client "disconnect" to get get logged. The internal client is expected to be well behaved and unless there's an error I wouldn't think normal operations would ever disconnect before consuming the entire response from the app.

But it turns out I was wrong.

When using the InternalClient.iter_* methods, the common base method will break without iterating the response if it encounters a non 200 response (from _iter_items):

        while True:
            resp = self.make_request(
                'GET', '%s?format=json&marker=%s&end_marker=%s' %
                (path, quote(marker), quote(end_marker)),
                {}, acceptable_statuses)
            if not resp.status_int == 200:
                break

So the response from the application ("<html><h1>Not Found</h1><p>The resource could not be found.</p></html>") is never "read" and the proxy logs it as a disconnect instead of a 404.

I'm not so sure that's a bad behavior in the general case - but it wasn't expected (again, for me at least), and not terribly useful if the response was something interesting like a 503

Maybe InteralClient should be "fixed"?

John Dickinson (notmyname) on 2014-09-26

Changed in swift:
status:	New → Confirmed

Pradeep Kumar Singh (pradeep-singh-u) on 2015-05-14

Changed in swift:
assignee:	nobody → pradeep kumar singh (pradeep-singh-u)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-05-17: Fix proposed to swift (master)

#3

Fix proposed to branch: master
Review: https://review.openstack.org/183899

Changed in swift:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-11-06: Fix merged to swift (master)

#4

Reviewed: https://review.openstack.org/183899
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=37337d5fcb0a29344381f8de7d810a9e76e10928
Submitter: Jenkins
Branch: master

commit 37337d5fcb0a29344381f8de7d810a9e76e10928
Author: Pradeep Kumar Singh <email address hidden>
Date: Sun May 17 13:35:58 2015 +0530

Read the response body, if response status is greater than 300.

    internal_client was not reading response if response status is not 200.
    So proxy server treats this as client disconnect and logs 499 in log file.
    This patch fixes that by reading response if response status is greater
    than or equal to 300 and in acceptable statuses.

Closes-Bug: #1364752

Change-Id: I0512a25895da583956f76031e3c5de5c970bce01

Changed in swift:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-11-11: Fix proposed to swift (feature/crypto)

#5

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/244249

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-11-11: Fix merged to swift (feature/crypto)

#6

Download full text (24.4 KiB)

Reviewed: https://review.openstack.org/244249
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=19226e4449bca399981cde3a1e9882735fed74d5
Submitter: Jenkins
Branch: feature/crypto

commit 2d85a3f6997fbadac210207f841c4ec25ff50cc4
Author: Tushar Gohad <email address hidden>
Date: Fri Oct 23 13:16:33 2015 +0000

EC: Use best available ec_type in unittests

    To minimize external library dependencies for Swift unit
    tests and SAIO, PyECLib 1.1.1 introduces a native backend
    'liberasurecode_rs_vand.' This patch is to migrate over
    the unit tests to the new ec_type when available.

This change will work with current pyeclib requirements
(==1.0.7) and also future requirements (>=1.0.7).

    When we're able to raise *our* requirements to >=1.1.1 we
    should remove jerasure from the list of preferred backends.
    Related SAIO doc and example config changes should be
    included with that patch.

Co-Authored-By: Clay Gerrard <email address hidden>
Change-Id: Idf657f0acf0479bc8158972e568a29dbc08eaf3b

commit 66dc1eebb1db51c81891b1f7bcc3e85aef9b8c1d
Author: Bill Huber <email address hidden>
Date: Fri Oct 16 11:27:34 2015 -0500

ObjectControllers return application errors as 499 on bad read

    In the _transfer_data method, we translate all (Exception, Timeout)
    into a 499 whereas we should consider translating them to 500 on
    particular returning error scenarios.

This affects both ReplicatedObjectController and ECObjectControllear.

Change-Id: I571bbc5b1451243907b094a5718c8735fd824268
Closes-Bug: 1504299

commit 705642db4c27c34ac8eb840d40b16470b85418c8
Author: venkatamahesh <email address hidden>
Date: Tue Nov 10 19:46:31 2015 +0530

Change stackforge repo to openstack repo

The projects which are moved to openstack
are corrected by replacing stackforge with openstack

Change-Id: I65b794a7f10df617bc2a4caf2c4010477a82dbc2

commit d755f5b52024e255787d180bbca60c09192a8850
Author: John Dickinson <email address hidden>
Date: Mon Nov 9 22:03:34 2015 -0800

suppress warning output in a unit test

    test_write_builder_after_device_removal() wasn't setting a
    default min_part_hours so a warnign was printed. Explicitly
    adding a min_part_hours suppresses the warning

Change-Id: I6f234b72c34e066abb91f28e6eacf50e29be8842

commit 37337d5fcb0a29344381f8de7d810a9e76e10928
Author: Pradeep Kumar Singh <email address hidden>
Date: Sun May 17 13:35:58 2015 +0530

Read the response body, if response status is greater than 300.

    internal_client was not reading response if response status is not 200.
    So proxy server treats this as client disconnect and logs 499 in log file.
    This patch fixes that by reading response if response status is greater
    than or equal to 300 and in acceptable statuses.

Closes-Bug: #1364752

Change-Id: I0512a25895da583956f76031e3c5de5c970bce01

commit c81f202f714c36390a25b24fa96d1b277788ade9
Author: Kota Tsuyuzaki <email address hidden>
Date: Thu Nov 5 17:37:30 2015 -0800

Add missing docs for ring.builder.rebalance

...

Reviewed:  https://review.openstack.org/244249
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=19226e4449bca399981cde3a1e9882735fed74d5
Submitter: Jenkins
Branch:    feature/crypto

commit 2d85a3f6997fbadac210207f841c4ec25ff50cc4
Author: Tushar Gohad <tushar.gohad@intel.com>
Date:   Fri Oct 23 13:16:33 2015 +0000

EC: Use best available ec_type in unittests
    
    To minimize external library dependencies for Swift unit
    tests and SAIO, PyECLib 1.1.1 introduces a native backend
    'liberasurecode_rs_vand.'  This patch is to migrate over
    the unit tests to the new ec_type when available.
    
    This change will work with current pyeclib requirements
    (==1.0.7) and also future requirements (>=1.0.7).
    
    When we're able to raise *our* requirements to >=1.1.1 we
    should remove jerasure from the list of preferred backends.
    Related SAIO doc and example config changes should be
    included with that patch.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Change-Id: Idf657f0acf0479bc8158972e568a29dbc08eaf3b

commit 66dc1eebb1db51c81891b1f7bcc3e85aef9b8c1d
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Fri Oct 16 11:27:34 2015 -0500

ObjectControllers return application errors as 499 on bad read
    
    In the _transfer_data method, we translate all (Exception, Timeout)
    into a 499 whereas we should consider translating them to 500 on
    particular returning error scenarios.
    
    This affects both ReplicatedObjectController and ECObjectControllear.
    
    Change-Id: I571bbc5b1451243907b094a5718c8735fd824268
    Closes-Bug: 1504299

commit 705642db4c27c34ac8eb840d40b16470b85418c8
Author: venkatamahesh <venkatamaheshkotha@gmail.com>
Date:   Tue Nov 10 19:46:31 2015 +0530

Change stackforge repo to openstack repo
    
    The projects which are moved to openstack
    are corrected by replacing stackforge with openstack
    
    Change-Id: I65b794a7f10df617bc2a4caf2c4010477a82dbc2

commit d755f5b52024e255787d180bbca60c09192a8850
Author: John Dickinson <me@not.mn>
Date:   Mon Nov 9 22:03:34 2015 -0800

suppress warning output in a unit test
    
    test_write_builder_after_device_removal() wasn't setting a
    default min_part_hours so a warnign was printed. Explicitly
    adding a min_part_hours suppresses the warning
    
    Change-Id: I6f234b72c34e066abb91f28e6eacf50e29be8842

commit 37337d5fcb0a29344381f8de7d810a9e76e10928
Author: Pradeep Kumar Singh <pradeep.singh@nectechnologies.in>
Date:   Sun May 17 13:35:58 2015 +0530

Read the response body, if response status is greater than 300.
    
    internal_client was not reading response if response status is not 200.
    So proxy server treats this as client disconnect and logs 499 in log file.
    This patch fixes that by reading response if response status is greater
    than or equal to 300 and in acceptable statuses.
    
    Closes-Bug: #1364752
    
    Change-Id: I0512a25895da583956f76031e3c5de5c970bce01

commit c81f202f714c36390a25b24fa96d1b277788ade9
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu Nov 5 17:37:30 2015 -0800

Add missing docs for ring.builder.rebalance
    
    commit 71993d84e88cc1d5f7742182905cace21c7e88cb added
    a new 'remove_dev' column to the
    swift.common.ring.builder.rebalance return value.
    
    This patch adds the docs for that and clean up a bit to
    the variable name to be easy to read.
    
    Change-Id: Idfd46e47b9f6894cbafc8b7701a4c7414212f79f

commit 331382e1922add01959b88e58bc8c1d2029f4550
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Thu Oct 29 21:09:01 2015 +0000

Add unit tests to cover print_item_locations
    
    Add unit tests to cover all code paths in print_item_locations
    function in cli/info.py.
    
    Update comment to match what's tested for invalid/missing policy.
    
    Update tests to verify output of print_item_locations
    
    Corrected PEP8 compliance violations.
    
    Change-Id: I84958cb70205ee8d7ea246826dd56201fa642da9

commit e0430fc74a564a0556ef5eab1accd110566aee31
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Sun Nov 1 18:10:06 2015 +0100

Compare Swift config checksum in swift-recon --all
    
    Change-Id: I796fe0895f4e5ddeb04c0d79a73579ce8bb9aa40

commit 609095b23f6c5734c6a7aed80c488b96b31137e8
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Aug 27 00:51:58 2015 +0200

On py3, use dnspython3 dependency, not dnspython
    
    Update requirements to install dnspython3 on Python 3 and dnspython on
    Python 2. dnspython is not compatible with Python 3, and dnspython3
    is not compatible with Python 2.
    
    Add a requirement on pbr to ensure that pbr 1.6 or newer is
    installed. pbr 1.0 or newer is required to support environment
    markers in requirements which are now used to select dnspython or
    dnspython3 depending on the Python major version.
    
    Note: pbr 1.0 is enough, but OpenStack global requirements asks to
    use "pbr>=1.6".
    
    Change-Id: Ie5d437cd396972d4143ffffb96ec3c289e221b08

commit 6858510b59c23ebef527a92c3258dc3c5c6046f5
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Thu Sep 3 16:13:17 2015 +0100

Re-organise ssync tests
    
    We have some tests that exercise both the sender and receiver,
    but are spread across test_ssync_sender.py and test_ssync_receiver.py.
    This creates a new module test_ssync.py and moves the end-to-end tests
    into there.
    
    Change-Id: Iea3e9932734924453f7241432afda90abbc75c06

commit 48421f902e2e293a3c9ef672dbec5971ca530fd6
Author: Venkateswarlu Pallamala <p.venkatesh551@gmail.com>
Date:   Wed Nov 4 00:53:05 2015 -0800

Optimize the code performance
    
    Usually, the number of characters in FORBIDDEN_CHARS are fewer than
    the number of characters in the req.path - so it's better to use
    smaller count for the outer loop.
    
    Change-Id: I78b7e0ff4cae1cc90e8d29d7bddcec7fe0ba4bd4

commit faef717cd3fda6aeb3a0799f25d0483bc9bdce7a
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Thu Oct 22 10:14:29 2015 -0500

Add unit tests for swift.proxy.controllers.base
    
    This patch adds more unit tests to diminish missing pieces
    of the coverage in the proxy_controllers_base unit test.
    
    Change-Id: I85ba1955c681cc9d5b2a70ac31155678d2e5b6fd

commit febdd6c1b4035a3b63b5deb7690b2a41014e5321
Author: Peter Lisák <peter.lisak@firma.seznam.cz>
Date:   Fri Oct 23 12:40:15 2015 +0200

Device marked to be removed in info about the ring.
    
    Showing devices with 'DEL' mark if will be removed next rebalance.
    
    Change-Id: I171aa8658b1c4ac1689ab9532fe65d114567baa7

commit a4a5fcad8f1ec9059cfab0c63e81106355b0dbd5
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Oct 7 02:14:49 2015 -0700

Add unit tests for utils.validate_hash_path
    
    This patch add some unit tests to prevent regression and to describe
    validate_hash_path behavior, I found in review with
    
    https://review.openstack.org/#/c/231864/
    
    *bonus*
    - Fix test_hash_path to use "with" syntax instead of "try/finally"
      for assigning a testing value into a global variable.
    
    Change-Id: I948999a8fb8addb9a378dbf8bee853b205aeafad

commit e31ecb24b6684da692fbca8b7fe34c1dd4047a3a
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Oct 21 13:05:37 2015 -0700

Get rid of contextlib.nested() for py3
    
    contextlib.nested() is missing completely in Python 3.
    
    Since 2.7, we can use multiple context managers in a 'with' statement,
    like so:
    
        with thing1() as t1, thing2() as t2:
            do_stuff()
    
    Now, if we had some code that needed to nest an arbitrary number of
    context managers, there's stuff we could do with contextlib.ExitStack
    and such... but we don't. We only use contextlib.nested() in tests to
    set up bunches of mocks without crazy-deep indentation, and all that
    stuff fits perfectly into multiple-context-manager 'with' statements.
    
    Change-Id: Id472958b007948f05dbd4c7fb8cf3ffab58e2681

commit 73e032049ffb9b371f30d5230472b920446d08f4
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Fri Oct 23 18:20:25 2015 +0000

Update admin guide with region.
    
    Added region prefix to example commands for adding devices to ring.
    Also updates description to include region prefix.
    
    Change-Id: Ie6d6485b497cea973e37909b5b19b44946c8aa89

commit 5d8d2e98ee8004cb009f4bd419fbd8cd5757aaa2
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Fri Oct 23 16:38:24 2015 +0100

Minor improvement to updater unit tests
    
    Follow up for change I7c57bba053711a27d3802efe6f2a0bf53483a54f
    
    The current tests won't prevent this kind of regression in the
    updaters' __init__ methods:
    
      self.node_timeout = int(float(conf.get('node_timeout', 10)))
    
    so this makes them a little tighter.
    
    Change-Id: I60bd79ac392deb1e8a108357d48474dddd9028c1

commit 1a2b54fc0a22713ff0ca575e0a22834682670c55
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Mon Oct 19 13:55:02 2015 +0100

Fix missing *-replicator conf sections in deployment guide
    
    The doc for these sections was missing because of an rst error - the
    source is there in rst file but didn't make it into the html output.
    
    Add doc for per_diff and max_diffs in account and container doc sections.
    
    Also, fix a bunch of other sphinx build errors and most of the warnings.
    
    Change-Id: If9ed2619b2f92c6c65a94f41d8819db8726d3893

commit 8f78c1a2b066e816cfd643d048ebbe3b654fbe87
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Fri Oct 23 06:34:15 2015 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Ifebb8f1b50ac4c2c0f43df48c2414ce0f3742e42

commit 59849a9923d7e214725afc14d0d229194884fc54
Author: Zack M. Davis <zdavis@swiftstack.com>
Date:   Thu Oct 22 15:13:09 2015 -0700

SLO docs(tring): manifest params in table, code block; copyedits
    
    It's æsthetically pleasing to display the segment description parameters
    in a table (using the reStructuredText "simple" table markup) and to use
    a code block (which receives syntax hightlighting in the rendered
    document!) for the example JSON fragment. Also, a few very minor copy
    changes are made (a comma after an introductory phrase clarifies; JSON
    and ETag receieve their canonical capitalizations; the elements of the
    manifest JSON aren't "files" exactly; the inclusive nature of the range
    is noted for the benefit of harried readers who won't read the "Range
    Specification" section later in the document).
    
    Change-Id: I5296fc04c6fe5410ad14bba7de0a7f8ab060b126

commit b3a0acd5c77201161dcd169850df589ede331106
Author: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
Date:   Tue Oct 20 15:16:39 2015 +0000

DiskFileNoSpace check during object delete
    
    On a full disk, a call to delete an object will fail when it tries to
    write tombstones. Handling DiskFileNoSpace exception raised by
    swift.common.utils.
    
    Change-Id: I8f0cfcc4159ee154fcd3e7ca90c422aa5aadf0b3
    Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
    Closes-Bug: 1491675

commit b376e9dc342a2ea4427467b7cd276f335298134d
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Oct 21 05:33:04 2015 -0700

Add verification socket close causes StopIteration
    
    This patch add a expositive test that closing connection
    (fd and socket) causes StopIteration on reader at proxy server
    on PUT object case.
    
    And then, if the connection closed and proxy detects a lack of
    data length, it will result in HTTPClientDisconnected.
    
    The only reason why I leave test_client_disconnect as current
    is Bill is now working for the change at
    
    https://review.openstack.org/#/c/236007/
    
    Change-Id: If68f752a2b0dd6cfad4d007ca5d155e6ffb378e5

commit 71993d84e88cc1d5f7742182905cace21c7e88cb
Author: Lisak, Peter <peter.lisak@firma.seznam.cz>
Date:   Fri Oct 9 16:13:55 2015 +0200

swift-ring-builder can't remove a device with zero weight
    
    If a device with 0 weight is tried to remove, the following rebalance
    does not write changes into builder file.
    
    Scenario:
    $ swift-ring-builder object.builder set_weight --id 1 0.00
    $ swift-ring-builder object.builder rebalance
    Wait for moving files out of the device id=1.
    $ swift-ring-builder object.builder remove --id 1
    $ swift-ring-builder object.builder rebalance
    In fact, the device id=1 is not removed after rebalance (must be --force used).
    
    Change-Id: Iad5a444023eae9882a3addd7f119ff4d18559ddd

commit 2135841e6e3fa6705562a72030c31f0c69fd4470
Author: asettle <alexandra.settle@rackspace.com>
Date:   Wed Oct 7 13:55:45 2015 +1000

Documenting the ability to specify ranges for SLO segments
    
    Change-Id: Ied854f6385f3b28975808f4707e531581bf6c18e
    Closes-bug: 1499152

commit b6b7578190bc5de266d08c22984622d5566b8456
Author: Lisak, Peter <peter.lisak@firma.seznam.cz>
Date:   Tue Oct 13 17:00:41 2015 +0200

node_timeout as float in configs
    
    It is more convenient to use float node_timeout for fine tunning latency.
    
    Change-Id: I7c57bba053711a27d3802efe6f2a0bf53483a54f

commit 71e573f4ba8a45dd5962f9d180c6236ce85627ca
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Oct 19 16:59:14 2015 +0200

Port swift.common.utils.backward() to Python 3
    
    backward() is written to handle binary files:
    
    * Replace literal native strings to literal byte strings: add b'...'
      prefix
    * Update unit tests: use byte strings
    * TemporaryFile(): use the default mode 'w+b', instead of using 'r+w'
      ('r+w' mode creates a Unicode file on Python 3).
    
    Change-Id: Ic91f7e6c605db0b888763080d49f0f501029837f

commit 4a13dcc4a8d7c890ff322190ca80fbb460b55de4
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Fri Oct 9 17:25:08 2015 +1100

Make db_replicator usync smaller containers
    
    The current rule inside the db_replicator is to rsync+merge
    containers during replication if the difference between rowids
    differ by more than 50%:
    
      # if the difference in rowids between the two differs by
      # more than 50%, rsync then do a remote merge.
      if rinfo['max_row'] / float(info['max_row']) < 0.5:
    
    This mean on smaller containers, that only have few rows, and differ
    by a small number still rsync+merge rather then copying rows.
    
    This change adds a new condition, the difference in the rowids must
    be greater than the defined per_diff otherwise usync will be used:
    
      # if the difference in rowids between the two differs by
      # more than 50% and the difference is greater than per_diff,
      # rsync then do a remote merge.
      # NOTE: difference > per_diff stops us from dropping to rsync
      # on smaller containers, who have only a few rows to sync.
      if rinfo['max_row'] / float(info['max_row']) < 0.5 and \
              info['max_row'] - rinfo['max_row'] > self.per_diff:
    
    Change-Id: I9e779f71bf37714919a525404565dd075762b0d4
    Closes-bug: #1019712

commit a9ddc2d9ea402eaac7ccd8992387f77855968ab5
Author: Mahati Chamarthy <mahati.chamarthy@gmail.com>
Date:   Fri Oct 16 18:18:33 2015 +0530

Hyperlink fix to first contribution doc
    
    Change-Id: I19fc1abc89f888233b80a57c68a152c1c1758640

commit 83a1151d13e096b480aefe6ec18259f2d7d021db
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Fri Oct 9 16:45:20 2015 -0600

Interpolate the explanation string not whole HTML body
    
    The only reason this exists is that I promised to do it.
    But in our case, there's no big advantage, and here's why.
    
    The general thinking goes that strings must be interpolated
    first because the body may contain a syntax that confuses the
    interpolation. So this patch makes the code "more correct".
    However, our HTML template is tightly controlled. It's not
    like it contains additional percents.
    
    So I'll just leave this here for now while I'm asking if
    the content type is set correctly.
    
    Change-Id: Ia18aeb0f94ef389f8b95450986a566e5fa06aa10

commit 43cdc29669741568a898dc87eee3e09280887bba
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Tue Oct 13 00:22:31 2015 -0700

Add backend subrequest tests for staticweb
    
    This is an followup for
    https://review.openstack.org/#/c/227204
    
    Above patch changes staticweb to use just make_env instead of
    pre authed env. The tests included in this patch assert
    "backend requests generated by staticweb *NEVER* include pre
    authed information (i.e. authorize_override=True)".
    
    Change-Id: Ifbbc43d3b5ac114bc3d5e4fcab9cacdf9a3f1a5c

commit 9993c1f124bcb6d645254196fe43d8c4f9feae6e
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Oct 14 15:49:35 2015 +0100

Add test to verify staticweb uses auth'd subrequests
    
    Follow on for change Icf159d7e567ac5481e710c5910db686bdcba6336
    
    Related-Bug: 1489749
    
    Change-Id: I96920645d50322c5232bbcd6bb38b0f26795b92d

commit 4eb796b29ddee52f5010655e0d5477f2d7923854
Author: Carlos Cavanna <ccavanna@ca.ibm.com>
Date:   Thu Oct 8 13:40:20 2015 -0400

bug/1270114. Missing documentation for Bulk Middleware.
    
    Closes-Bug: #1270114
    
    This patch addresses the missing bulk middleware documentation for
    the following bug:
    
    https://bugs.launchpad.net/swift/+bug/1270114
    
    And feature:
    
    https://review.openstack.org/#/c/176161/
    
    The documentation will show under:
    
    http://docs.openstack.org/developer/swift/middleware.html
    
    Change-Id: I73c101e09830d7120b5e5c1333ae8d570a9e2303

commit 384b91eb824376659989b904f9396cbf2e02d2bd
Author: asettle <alexandra.settle@rackspace.com>
Date:   Thu Sep 3 15:11:46 2015 +1000

Moving DLO functionality doc to the middleware code
    
    This change moves the RST DLO documentation from
    statically inside overview_large_objects.rst and moves it
    to middleware/dlo.py.
    This is where all middleware RST documentation is defined.
    
    The overview_large_objects.rst is still the main page
    for information on large objects, so now dynamically
    points to both the DLO and SLO middleware RST
    documentation and the relevant middleware.rst page
    simply points to it.
    
    Change-Id: I40d918c8b7bc608ab945805d69fe359521df038a
    Closes-bug: #1276852

commit a7af802497f3b86d33cbee852500df456c316685
Author: Christian Schwede <cschwede@redhat.com>
Date:   Sat Aug 29 16:03:07 2015 +0000

Do not use pre-authenticated requests in staticweb
    
    staticweb middleware uses make_pre_authed_env, this makes it possible to
    anonymously list containers without any read acl set if the metadata
    "web-listings: true" is set on a container. Using make_env enforces correct
    read_acl validation; however it is now also required to add ".rlistings" to the
    read acl.
    
    Also, if the staticweb middleware is put in the proxy pipeline before an
    authentication middleware, it broke authenticated GET and HEAD requests. This
    has some side effects in clients, because a html response is sent that might be
    parsed wrongly by the client. In case of python-swiftclient this was shown as an
    empty container without any ACL or web-listings:true meta set. This might lead
    to information leaks, because a user trusts the output from python-swiftclient
    and assumes an empty, private container even if the container contains public
    readable data. staticweb now checks if "swift.authorize" is included in the
    environ and skips itself if not.
    
    Closes-Bug: 1489749
    Change-Id: Icf159d7e567ac5481e710c5910db686bdcba6336
    Depends-On: Ie24bb995023c377e49796910ad80a256b00daa03

commit 2996974e5d48b4efaa1b271b8fbd0387bced7242
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Sat Oct 10 14:56:30 2015 +0200

Script for running unit, func and probe tests at once
    
    When developing Swift it's often needed to run all tests.
    This script makes it much simpler.
    
    Change-Id: I67e6f7cc05ebd0475001c1b56e8f6fd09c8c644f

commit c2182fd4163050a5f76eb3dedb7703dc821fa83d
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Fri Jul 17 20:20:15 2015 +0530

Python3: do not use im_self/im_func/func_closure
    
    Use __self__, __func__ and __closure__ instead, as they work
    with both Python 2 and 3.
    
    Modifying usage of __func__ in codebase.
    
    Change-Id: I57e907c28c1d4646605e70194ea3650806730b83

commit c0866ceaac2f69ae01345a795520141f59ec64f5
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Fri Sep 25 17:26:37 2015 -0700

Improve SLO PUT error checking
    
    This commit tries to give the user a reason that their SLO manifest
    was invalid instead of just saying "Invalid SLO Manifest File". It
    doesn't get every error condition, but it's better than before.
    
    Examples of things that now have real error messages include:
     * bad keys in manifest (e.g. using "name" instead of "path")
     * bogus range (e.g. "bytes=123-taco")
     * multiple ranges (e.g. "bytes=10-20,30-40")
     * bad JSON structure (i.e. not a list of objects)
     * non-integer size_bytes
    
    Also fixed an annoyance with unspecified-size segments that are too
    small. Previously, if you uploaded a segment reference with
    '{"size_bytes": null, ...}' in it and the referenced segment was less
    than 1 MiB, you'd get a response that looked like this:
    
        HTTP/1.1 400 Bad Request
        Content-Length: 62
        Content-Type: text/html; charset=UTF-8
        X-Trans-Id: txd9ee3b25896642098e4d9-0055dd095a
        Date: Wed, 26 Aug 2015 00:33:30 GMT
    
        Each segment, except the last, must be at least 1048576 bytes.
    
    This is true, but not particularly helpful, since it doesn't tell you
    which of your segments violated the rule.
    
    Now you get something more like this:
    
        HTTP/1.1 400 Bad Request
        Content-Length: 49
        Content-Type: text/plain
        X-Trans-Id: tx586e52580bac4956ad8e2-0055dd09c2
        Date: Wed, 26 Aug 2015 00:35:14 GMT
    
        Errors:
        /segs/small, Too Small; each segment, except the last...
    
    It's not exactly a tutorial on SLO manifests, but at least it names
    the problematic segment.
    
    This also changes the status code for a self-referential manifest from
    409 to 400. The rest of the error machinery was using 400, and
    special-casing self-reference would be really annoying. Besides, now
    that we're showing more than one error message at a time, what would
    the right status code be for a manifest with a self-referential
    segment *and* a segment with a bad range? 400? 409? 404.5? It's much
    more consistent to just say invalid manifest --> 400.
    
    Change-Id: I2275683230b36bc273319254e37c16b9e9b9d69c

commit 84f0a544452fefc65f6755fdd04cc2cc1eb11c88
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Oct 8 15:42:27 2015 +0200

py3: Replace basestring with six.string_types
    
    The builtin basestring type was removed in Python 3. Replace it with
    six.string_types which works on Python 2 and Python 3.
    
    Change-Id: Ib92a729682322cc65b41050ae169167be2899e2c

commit 2447e0cf6ed502375a80e33e6aa4a5110305fa6f
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Oct 8 15:41:02 2015 +0200

Replace itertools.ifilter with six.moves.filter
    
    Replace itertools.ifilter() with six.moves.filter(). The
    itertools.ifilter was removed in Python 3. This change makes modified
    code compatible with Python 2 and Python 3.
    
    Replace itertools.izip() with six.moves.zip().
    
    The patch was generated by the itertools operation of the sixer tool.
    
    Change-Id: Ie7f787cc6d66edfceb8fa2c1a906351a8c8c5fed

commit 752ceb266b3e72361242250e50396671df8b9595
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Oct 2 16:40:28 2015 -0700

Close ECAppIter's sub-generators before propagating GeneratorExit
    
    ... which ensures no Timeouts remain pending after the parent generator
    is closed when a client disconnects before being able to read the entire
    body.
    
    Also tighten up a few tests that may have left some open ECAppIter
    generators lying about after the tests themselves had finished.  This
    has the side effect of preventing the extraneous printing of the Timeout
    errors being raised by the eventlet hub in the background while our
    unittests are running.
    
    Change-Id: I156d873c72c19623bcfbf39bf120c98800b3cada

tags:

added: in-feature-crypto

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-01-07: Fix proposed to swift (feature/hummingbird)

#7

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/264517

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-01-08: Fix merged to swift (feature/hummingbird)

#8

Download full text (54.1 KiB)

Reviewed: https://review.openstack.org/264517
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=93ddaffaebb620bc712d52d0267194562dce02be
Submitter: Jenkins
Branch: feature/hummingbird

commit f53cf1043d078451c4b9957027bf3af378aa0166
Author: Ondřej Nový <email address hidden>
Date: Tue Jan 5 20:20:15 2016 +0100

Fixed few misspellings in comments

Change-Id: I8479c85cb8821c48b5da197cac37c80e5c1c7f05

commit 79222e327f9df6335b58e17a6c8dd0dc44b86c17
Author: ChangBo Guo(gcb) <email address hidden>
Date: Sat Dec 26 13:13:37 2015 +0800

Fix AttributeError for LogAdapter

LogAdapter object has no attribute 'warn' but has attribute
'warning'.

Closes-Bug: #1529321
Change-Id: I0e0bd0a3dbc4bb5c1f0b343a8809e53491a1da5f

commit 684c4c04592278a280032002b5313b171ee7a4c0
Author: janonymous <email address hidden>
Date: Sun Aug 2 22:47:42 2015 +0530

Python 3 deprecated the logger.warn method in favor of warning

DeprecationWarning: The 'warn' method is deprecated, use 'warning'
instead

Change-Id: I35df44374c4521b1f06be7a96c0b873e8c3674d8

commit d0a026fcb8e8a9f5475699cc56e1998bdc4cd5ca
Author: Hisashi Osanai <email address hidden>
Date: Wed Dec 16 18:50:37 2015 +0900

Fix duplication for headers in Access-Control-Expose-Headers

There are following problems with Access-Control-Expose-Headers.

    * If headers in X-Container-Meta-Access-Control-Expose-Headers are
      configured, the headers are kept with case-sensitive string.
      Then a CORS request comes, the headers are merged into
      Access-Control-Expose-Headers as case-sensitive string even if
      there is a same header which is not case-sensitive string.

    * Access-Control-Expose-Headers is handled by a list.
      If X-Container/Object-Meta-XXX is configured in container/object
      and X-Container-Meta-Access-Control-Expose-Headers, same header
      is listed in Access-Control-Expose-Headers.

This patch provides a fix for the problems.

Change-Id: Ifc1c14eb3833ec6a851631cfc23008648463bd81

commit 0bcd7fd50ec0763dcb366dbf43a9696ca3806f15
Author: Bill Huber <email address hidden>
Date: Fri Nov 20 12:09:26 2015 -0600

Update Erasure Coding Overview doc to remove Beta version

    The major functionality of EC has been released for Liberty and
    the beta version of the code has been removed since it is now
    in production.

Change-Id: If60712045fb1af803093d6753fcd60434e637772

commit 84ba24a75640be4212e0f984c284faf4c894e7c6
Author: Alistair Coles <email address hidden>
Date: Fri Dec 18 11:24:34 2015 +0000

Fix rst errors so that html docs are complete

    rst table format errors don't break the gate job
    but do cause sections of the documents to go missing
    from the html output.

Change-Id: Ic8c9953c93d03dcdafd8f47b271d276c7b356dc3

commit 87f7e907ee412f5847f1f9ffca7a566fb148c6b1
Author: Matthew Oliver <email address hidden>
Date: Wed Dec 16 17:19:24 2015 +1100

Pass HTTP_REFERER down to subrequests

Currently a HTTP_REFERER (Referer) header isn't passed down to
subreq...

Reviewed:  https://review.openstack.org/264517
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=93ddaffaebb620bc712d52d0267194562dce02be
Submitter: Jenkins
Branch:    feature/hummingbird

commit f53cf1043d078451c4b9957027bf3af378aa0166
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Tue Jan 5 20:20:15 2016 +0100

Fixed few misspellings in comments
    
    Change-Id: I8479c85cb8821c48b5da197cac37c80e5c1c7f05

commit 79222e327f9df6335b58e17a6c8dd0dc44b86c17
Author: ChangBo Guo(gcb) <eric.guo@easystack.cn>
Date:   Sat Dec 26 13:13:37 2015 +0800

Fix AttributeError for LogAdapter
    
    LogAdapter object has no attribute 'warn' but has attribute
    'warning'.
    
    Closes-Bug: #1529321
    Change-Id: I0e0bd0a3dbc4bb5c1f0b343a8809e53491a1da5f

commit 684c4c04592278a280032002b5313b171ee7a4c0
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sun Aug 2 22:47:42 2015 +0530

Python 3 deprecated the logger.warn method in favor of warning
    
    DeprecationWarning: The 'warn' method is deprecated, use 'warning'
    instead
    
    Change-Id: I35df44374c4521b1f06be7a96c0b873e8c3674d8

commit d0a026fcb8e8a9f5475699cc56e1998bdc4cd5ca
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Wed Dec 16 18:50:37 2015 +0900

Fix duplication for headers in Access-Control-Expose-Headers
    
    There are following problems with Access-Control-Expose-Headers.
    
    * If headers in X-Container-Meta-Access-Control-Expose-Headers are
      configured, the headers are kept with case-sensitive string.
      Then a CORS request comes, the headers are merged into
      Access-Control-Expose-Headers as case-sensitive string even if
      there is a same header which is not case-sensitive string.
    
    * Access-Control-Expose-Headers is handled by a list.
      If X-Container/Object-Meta-XXX is configured in container/object
      and X-Container-Meta-Access-Control-Expose-Headers, same header
      is listed in Access-Control-Expose-Headers.
    
    This patch provides a fix for the problems.
    
    Change-Id: Ifc1c14eb3833ec6a851631cfc23008648463bd81

commit 0bcd7fd50ec0763dcb366dbf43a9696ca3806f15
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Fri Nov 20 12:09:26 2015 -0600

Update Erasure Coding Overview doc to remove Beta version
    
    The major functionality of EC has been released for Liberty and
    the beta version of the code has been removed since it is now
    in production.
    
    Change-Id: If60712045fb1af803093d6753fcd60434e637772

commit 84ba24a75640be4212e0f984c284faf4c894e7c6
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Fri Dec 18 11:24:34 2015 +0000

Fix rst errors so that html docs are complete
    
    rst table format errors don't break the gate job
    but do cause sections of the documents to go missing
    from the html output.
    
    Change-Id: Ic8c9953c93d03dcdafd8f47b271d276c7b356dc3

commit 87f7e907ee412f5847f1f9ffca7a566fb148c6b1
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Wed Dec 16 17:19:24 2015 +1100

Pass HTTP_REFERER down to subrequests
    
    Currently a HTTP_REFERER (Referer) header isn't passed down to
    subrequests. This means *LO subrequests to segment containers
    return a 403 on a *LO GET when accessed by requests using referer
    ACLs.
    Currently the only way around referer access to *LO's is to make the
    segments container world readable.
    
    This change makes sure the referer header is passed into subrequests
    allowing a segments container to only need to be locked down with
    the same referer as the *LO container.
    
    This is a 1 line change to code, but also adds a unit and 2 functional
    functional tests (one for DLO and one for SLO).
    
    Change-Id: I1fa5328979302d9c8133aa739787c8dae6084f54
    Closes-Bug: #1526575

commit e15960a5d86e00a7d420edc4af034b27da0af8fd
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Dec 17 12:08:45 2015 +0000

Fix incorrect kwarg in auth middleware example
    
    When calling memcache_client.set(), timeout was deprecated
    and is now removed as a keyword arg, use time instead.
    
    Change-Id: Iedbd5b064853ef2b386963246f639fbcd3931cd3

commit 169a7c7f9e12ebc9933bd9ca4592e13b0de8b47b
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Dec 16 15:28:25 2015 +0000

Fix func test --until-failure and --no-discover options
    
    This patch changes functional test classes to subclass
    unittest2.TestCase rather than unittest.TestCase.
    This fixes errors when attempting to use
    
    tox -e func -- -n <test_path_including_test_method>
    
    and
    
    tox -e func -- --until-failure
    
    Also migrate from using nose.SkipTest to unittest2.SkipTest
    
    Change-Id: I903033f5e01833550b2f2b945894edca4233c4a2
    Closes-Bug: 1526725
    Co-Authored-By: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

commit b68311db95860ac1cab585a5ab66bd3b3abb765e
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Tue Dec 15 18:55:41 2015 -0800

Fix reconciler test to calc lastmodified as UTC
    
    Swift reconciler calculates the last-modified date as UTC but
    current test calculates it as local time zone. It triggers unit
    test failure in non-UTC environment.
    
    This patch fixes the test to calculate the last-modified as UTC
    as well.
    
    Change-Id: Ia0053f350daf2cb8c61ac01a933924b6e4b0cb37
    Closes-Bug: #1526588

commit 1bb665331af92422290fb585de7cb6a2497236e6
Author: Venkateswarlu Pallamala <p.venkatesh551@gmail.com>
Date:   Mon Nov 9 19:22:38 2015 -0800

remove unused parameters in the method
    
    make the helper methods as private by using convention
    
    Change-Id: I73b9604f8d5a0e85d012aac42b7963b618f5ad97

commit 9d7f71d5754c8b45f8e7c6ab80202de09933afb8
Author: Richard Hawkins <hurricanerix@gmail.com>
Date:   Fri Aug 7 18:14:13 2015 -0500

Modify functional tests to use ostestr/testr
    
    Defcore uses Tempest, which uses Test Repository.
    This change makes it easier for Defcore to pull functional
    tests from Swift and run them.  Additionally, using testr
    allows tests to be run in parallel.
    
    Concurrency set to 1 for now, >1 causes failures for
    reasons that are still TBD.
    
    With switch to ostestr all the server logs are being sent to stdout
    which makes it completely unreadable. Suppressing the logs by default
    now with a flag to enable it if desired.
    
    Co-Authored-By: John Dickinson <me@not.mn>
    Co-Authored-By: Robert Collins <rbtcollins@hpe.com>
    Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
    Co-Authored-By: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
    
    Change-Id: I53ef4a116996a772cf1f3abc2eb0ad60047322d5
    Related-Bug: 1177924

commit 2f4b79233e30d42140bbc07059417443bf7a0757
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Dec 15 15:49:42 2015 +0000

Minor cleanup of repeated identical test assertions
    
    assertDictContainsSubset is being called multiple times with
    same arguments in a loop. Since assertDictContainsSubset is
    deprecated form python 3.2, replace it with checks on
    individual key, value pairs.
    
    Change-Id: I7089487710147021f26bd77c36accf5751855d68

commit 60b2e02905d57f55169e506f4874b2334a1a68a5
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Mon Oct 5 16:15:29 2015 +0100

Make ECDiskFile report all fragments found on disk
    
    Refactor the disk file get_ondisk_files logic to enable
    ECDiskfile to gather *all* fragments found on disk (not just those
    with a matching .durable file) and make the fragments available
    via the DiskFile interface as a dict mapping:
    
        Timestamp --> list of fragment indexes
    
    Also, if a durable fragment has been found then the timestamp
    of the durable file is exposed via the diskfile interface.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Change-Id: I55e20a999685b94023d47b231d51007045ac920e

commit 450737f886050e486f518cdce0c97596ccad848d
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Tue Dec 15 11:33:56 2015 +0900

Fix a typo in development_auth.rst
    
    This patch uses correct name for "CORS".
    
    Change-Id: I5fee5c581a2b3adb7596a273baf05708bfa97f79

commit 40476ea0797690d3a90a9ed91906d26103dfa058
Author: John Dickinson <me@not.mn>
Date:   Mon Dec 14 10:52:22 2015 -0800

Document pretend_min_part_hours_passed
    
    Added a docstring for the swift-ring-builder CLI command
    "pretend_min_part_hours_passed". This is a dangerous operation, and
    that's why it hasn't been documented, but it can be useful at times.
    It should be made known to those who need it.
    
    Change-Id: I45bdbaacbbdda64c7510453e6d93e6d8563e3ecd

commit 6ade2908cca696ce1b48a7a19f4d460081fa5b0a
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Sun Dec 13 21:13:42 2015 +0100

Deprecated param timeout removed from memcached
    
    Change-Id: Idf042a79f0db148bf9f28a9e360cb2a3c18d385a

commit 52b534c9d13efde15841402dc86c99685334250f
Author: hgangwx <hgangwx@cn.ibm.com>
Date:   Fri Dec 11 17:31:36 2015 +0800

Fix some inconsistency in docstrings
    
    Added colon after ":returns" according to:
    http://docs.openstack.org/developer/hacking
    
     blank lines are added before the param lines
    
    Change-Id: I008bea270c6a4b7c80d1a84a2cb9fcc9b5a7264a

commit 88c9aed7c846402355a3c7831f34f3e833bbdf11
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Oct 19 16:19:28 2015 +0200

Port swift.common.utils.StatsdClient to Python 3
    
    * StatsdClient._send(): on Python 3, encode parts to UTF-8 and
      replace '|' with b'|' to join parts.
    * timing_stats(): replace func.func_name with func.__name__. The
      func_name attribute of functions was removed on Python 3, whereas
      the __name__ attribute is available on Python 2 and Python 3.
    * Fix unit tests to use bytes
    
    Change-Id: Ic279c9b54e91aabcc52587eed7758e268ffb155e

commit ca2dcc371921aa1aded6161287cc03c0940bf198
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Fri Dec 11 18:21:28 2015 +0100

Deprecated tox -downloadcache option removed
    
    Caching is enabled by default from pip version 6.0
    
    More info:
    https://testrun.org/tox/latest/config.html#confval-downloadcache=path
    https://pip.pypa.io/en/stable/reference/pip_install/#caching
    
    Change-Id: I9451a0f0dee5c5a3c0ca0a52f58bd353602661a2

commit 19c7dbc0ba49ff7e1f59190a7ead9ae9d3d80238
Author: Mahati Chamarthy <mahati.chamarthy@gmail.com>
Date:   Fri Dec 11 12:27:18 2015 +0530

Update versioned_writes doc
    
    Change-Id: Ibe53c79cf49330332112001c02a2a6b078764130

commit 211758f8cb02298fe16e59bf2954a146c6b24b83
Author: Catherine Northcott <catherine@northcott.nz>
Date:   Thu Nov 5 23:04:14 2015 +1300

Add support for storage policies to have more than one name
    
    This patch alters storage_policy.py to allow storage policies
    to have multiple names. Now users are able to add a number of
    human-readable aliases for storage policies. Policies now have
    a .name (the default name), .aliases (a string of comma
    seperated aliases), and .aliases_list (a list of all human
    readable names). Policies will always have an .aliases value
    if no aliases are set it will contain the default name.
    The policy docs and tests have been updated to reflect changes
    and policy.get_policy_info has been altered to display the
    name and aliases
    
    Change-Id: I02967ca8d7c790595e5ee551581196aa64552eea

commit 70bc3d1a3a240c35eba934010fdef99a775b8715
Author: John Dickinson <me@not.mn>
Date:   Tue Dec 8 16:08:04 2015 -0800

added a few ruby clients to associated projects
    
    Change-Id: I4764ba505646949ff694f8939947f83c6940128a

commit 73d0f1620a269f990dbd3d2796abf27e9a05e227
Author: Béla Vancsics <vancsics@inf.u-szeged.hu>
Date:   Tue Dec 8 10:17:08 2015 +0100

Not used parameter
    
    The account variable was not used in the method.
    
    Change-Id: I8e91d7616529f33b615bc52af76bfda01141d364

commit 7035639dfd239b52d4ed46aae50f78d16ec8cbfe
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Oct 15 16:20:58 2015 -0700

Put part-replicas where they go
    
    It's harder than it sounds.  There was really three challenges.
    
    Challenge #1 Initial Assignment
    ===============================
    
    Before starting to assign parts on this new shiny ring you've
    constructed, maybe we'll pause for a moment up front and consider the
    lay of the land.  This process is called the replica_plan.
    
    The replica_plan approach is separating part assignment failures into
    two modes:
    
     1) we considered the cluster topology and it's weights and came up with
        the wrong plan
    
     2) we failed to execute on the plan
    
    I failed at both parts plenty of times before I got it this close.  I'm
    sure a counter example still exists, but when we find it the new helper
    methods will let us reason about where things went wrong.
    
    Challenge #2 Fixing Placement
    =============================
    
    With a sound plan in hand, it's much easier to fail to execute on it the
    less material you have to execute with - so we gather up as many parts
    as we can - as long as we think we can find them a better home.
    
    Picking the right parts for gather is a black art - when you notice a
    balance is slow it's because it's spending so much time iterating over
    replica2part2dev trying to decide just the right parts to gather.
    
    The replica plan can help at least in the gross dispersion collection to
    gather up the worst offenders first before considering balance.  I think
    trying to avoid picking up parts that are stuck to the tier before
    falling into a forced grab on anything over parts_wanted helps with
    stability generally - but depending on where the parts_wanted are in
    relation to the full devices it's pretty easy pick up something that'll
    end up really close to where it started.
    
    I tried to break the gather methods into smaller pieces so it looked
    like I knew what I was doing.
    
    Going with a MAXIMUM gather iteration instead of balance (which doesn't
    reflect the replica_plan) doesn't seem to be costing me anything - most
    of the time the exit condition is either solved or all the parts overly
    aggressively locked up on min_part_hours.  So far, it mostly seemds if
    the thing is going to balance this round it'll get it in the first
    couple of shakes.
    
    Challenge #3 Crazy replica2part2dev tables
    ==========================================
    
    I think there's lots of ways "scars" can build up a ring which can
    result in very particular replica2part2dev tables that are physically
    difficult to dig out of.  It's repairing these scars that will take
    multiple rebalances to resolve.
    
    ... but at this point ...
    
    ... lacking a counter example ...
    
    I've been able to close up all the edge cases I was able to find.  It
    may not be quick, but progress will be made.
    
    Basically my strategy just required a better understanding of how
    previous algorithms were able to *mostly* keep things moving by brute
    forcing the whole mess with a bunch of randomness.  Then when we detect
    our "elegant" careful part selection isn't making progress - we can fall
    back to same old tricks.
    
    Validation
    ==========
    
    We validate against duplicate part replica assignment after rebalance
    and raise an ERROR if we detect more than one replica of a part assigned
    to the same device.
    
    In order to meet that requirement we have to have as many devices as
    replicas, so attempting to rebalance with too few devices w/o changing
    your replica_count is also an ERROR not a warning.
    
    Random Thoughts
    ===============
    
    As usual with rings, the test diff can be hard to reason about -
    hopefully I've added enough comments to assure future me that these
    assertions make sense.
    
    Despite being a large rewrite of a lot of important code, the existing
    code is known to have failed us.  This change fixes a critical bug that's
    trivial to reproduce in a critical component of the system.
    
    There's probably a bunch of error messages and exit status stuff that's
    not as helpful as it could be considering the new behaviors.
    
    Change-Id: I1bbe7be38806fc1c8b9181a722933c18a6c76e05
    Closes-Bug: #1452431

commit 62b9a0eeeaeb3a5a0daff981cf6fa1cb50e78a50
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Dec 7 10:19:51 2015 -0800

fix mock and assert in test_direct_client
    
    Change-Id: Ia6c14377e5dd768a5306a2448f0bf244ebc3634e

commit b63caf4521b0a4a32b9afdb6d247537848d0a8d6
Author: Tushar Gohad <tushar.gohad@intel.com>
Date:   Wed Nov 11 12:53:29 2015 -0700

Bump PyECLib requirement to >=1.0.7
    
    ... to match global-requirements.  This is an
    intermediate step in updating requirements to
    PyECLib>=1.1.1.
    
    Change-Id: I79f47fff6ec1adff214ca435f0d95aaf1ffd68f9

commit 73ded056aff0f25605fee2a633f3d13e0369606a
Author: Christian Schwede <cschwede@redhat.com>
Date:   Tue Dec 1 14:59:35 2015 +0000

Add functional test for repeated SLO segments
    
    Currently fails on master, passes with change
    If13af444ed301ebd8fd34a0d96a330ded648f0c4 applied.
    
    Change-Id: I05b231cc232d5b4117bccee40eebc7093114c61c

commit 679e6d5251c804cab8fd41d7a13e6bdd43967842
Author: John Dickinson <me@not.mn>
Date:   Mon Nov 30 11:58:40 2015 -0800

Add config sample reference to KeystoneAuth docs
    
    Change-Id: I38a7bd7b85de9d674386d75220b31cc467746da8

commit 5eaa5543c766536e48127d60e03cebe9c2824abb
Author: John Dickinson <me@not.mn>
Date:   Mon Nov 30 10:33:23 2015 -0800

add sample proxy pipeline for keystone integration
    
    Change-Id: I4b4fd9179d0234f001940e215c97d40a2a6204cd

commit 28c4b7310fead32f7ce073ee4bb503a450e521f1
Author: Peter Lisák <peter.lisak@firma.seznam.cz>
Date:   Tue Nov 24 16:33:35 2015 +0100

Unification of manpages and conf-samples (default values, etc)
    
    Change-Id: I47a3127ef698b4bd1537b1562901ee9c2b5924d4

commit 8e5b38b1ddb71d631c12368e6d917beb25d3a160
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Oct 7 16:47:11 2015 -0700

Expose tempurl's header restrictions via /info
    
    Also, clean up the module documentation a bit.
    
    Change-Id: Iaeb5eb264b118b78738187db9242540275e77444

commit 635bc7fa8f9a442b39af4955e98fcd2e2acf1e0a
Author: Ferenc Horváth <hferenc@inf.u-szeged.hu>
Date:   Wed Oct 7 11:10:28 2015 +0200

Replace string slicing with proper string methods
    
    Updated string prefix and suffix checker slicing to startswith()
    and endswith() methods.
    
    Using startswith() and endswith() improves readability, error-proneness
    and enhances maintainability.
    
    Change-Id: I1d5fbf116a61763346c6f92fd8023dbbe9bb37cf

commit 6c43bdc82bc27f1f60bd32c43fac6e8be562e136
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Nov 26 14:46:01 2015 +0000

Test for content-type params in multi-range response
    
    Updates the functional test to verify the fix applied by
    change Iff7274aa631a92cd7332212ed8b4378c27da4a1f
    
    Change-Id: Iae63ac027e4f4acfe46a36dc1325888b1f834ea4

commit b339e529c3464278b7ddd5670acd903070d1da1a
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Mar 25 14:59:43 2015 -0700

Add functional test for multi-range GET requests.
    
    Change-Id: I9d417faede707e4f3570074e410344cc8955007b

commit c28709f8f264f4426364b67a2a2755311111c1e4
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Nov 25 17:09:34 2015 -0800

Fix crash when a SLO repeats a segment.
    
    If a SLO manifest specified the same segment twice with no range on
    the latter, we would crash while trying to coalesce the segment
    specifications into a single request (for efficiency). Now we handle
    that missing range and do the right thing.
    
    Change-Id: If13af444ed301ebd8fd34a0d96a330ded648f0c4

commit ad722b3ed3ff1b92ebee829db3c289040960b24d
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Nov 25 18:49:39 2015 +0000

Include params in multipart message part content-type
    
    The content-type header inserted into a multipart message part
    is missing any params such as charset because its value is being
    fetched via the swob.Response content_type property, which conforms
    to webob spec and strips off all params.
    
    This was noticed in work on feature/crypto branch because the crypto
    meta param was being stripped off content-type in multipart messages,
    preventing the content-type being decrypted. But in general there is
    no suggestion in the multipart message spec [1] that params should
    not be included. In fact examples in [1] show the charset param
    included in the content-type value.
    
    To ensure that the multipart message part content-type includes the
    original content-type params, fetch it directly from the response
    headers.
    
    [1] http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
    
    Change-Id: Iff7274aa631a92cd7332212ed8b4378c27da4a1f

commit e06c8743941a27c1a3f50396385268b9e812c32e
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Nov 25 14:10:27 2015 +0000

Improve swift-init usage statement
    
    Clarify the behavior of strict and non-strict options.
    
    Define what <server> and <config> specify on the command line.
    
    Drive-by fix for missing param in search_tree docstring.
    
    Change-Id: I89fff88a088bc73001464b1fa8240e14a61a117d

commit 0e5cc89da56f550468512af6f3479382a1894815
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Oct 13 11:45:20 2015 -0700

Have versioned_writes use reverse listings
    
    This will reduce the overhead required for versioned containers in terms
    of memory, back-end requests, and wall time.
    
    Change-Id: I156216a14141db547de3cae1dcd5315ae31eaac2

commit aeccbc2074224c6688be6ef5aa8ba477c9631ba1
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Nov 24 14:47:30 2015 +0000

Add more functional tests for reverse listings
    
    Also fix typo in the api doc.
    
    Follow-on to change I5eb655360ac95042877da26d18707aebc11c02f6
    
    Change-Id: Ic1ea63c74a0a3b90a505865fba8fcfac584d0825

commit 41897d96a7dc3ddceccf4eed71345439b83eb243
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Nov 23 13:53:27 2015 -0800

Reverse-listings follow-up
    
     * With the end_prefix changes in the original commit, we no longer need
       the `or not name.startswith(prefix)` check.
     * Improve test coverage of reverse path listings.
    
    Change-Id: Iaa7d4b83647c3c150be95f88cb3cc9e4f0e33979

commit 7c1e6cd583debe43aca266643cd65f5103159dbf
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Thu Sep 11 16:51:51 2014 +1000

Add container and account reverse listings
    
    This change adds the ability to tell the container or account server to
    reverse their listings. This is done by sending a reverse=TRUE_VALUE,
    
    Where TRUE_VALUE is one of the values true can be in common/utils:
    
      TRUE_VALUES = set(('true', '1', 'yes', 'on', 't', 'y'))
    
    For example:
    
      curl -i -X GET -H "X-Auth-Token: $TOKEN" $STORAGE_URL/c/?reverse=on
    
    I borrowed the swapping of the markers code from Kevin's old change,
    thanks Kevin. And Tim Burke added some real nuggets of awesomeness.
    
    DocImpact
    Co-Authored-By: Kevin McDonald <kmcdonald@softlayer.com>
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    Implements: blueprint reverse-object-listing
    
    Change-Id: I5eb655360ac95042877da26d18707aebc11c02f6

commit 898223398189f96628db7d9928f146c5bb11ba88
Author: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
Date:   Fri Oct 30 21:30:51 2015 +0000

Unit tests for account/backend.py
    
    Add test for database create request without account
    Partial test for migrate call on database with storage_policy_index
    
    Change-Id: I7cfbd6bc7e2b341f433d88f600b19e54826a0e22
    Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

commit f80c97a6e964123e766c1e53d797a3b80433ac77
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Fri Nov 20 17:51:33 2015 +0000

Update .mailmap entry
    
    Fix/add entries for authors that are known to now have
    @hpe.com email addresses.
    
    Change-Id: I62c83b64e2378cb3d6ad33ac9e6ab111b8a8c86f

commit 16de32f1686a60fa5b3aa1802dcc42783846b05b
Author: Peter Lisák <peter.lisak@firma.seznam.cz>
Date:   Fri Nov 6 10:49:09 2015 +0100

Log error if a local device not identified in replicator
    
    Example:
    * Different port in config and in ring file.
    * Running daemon on server not in ring file.
    In both cases replication daemon is running but nothing is replicated.
    Error log helps to distinguish a local device can't be identified.
    
    Closes-Bug: 1508228
    Change-Id: I99351b7d9946f250b7750df91c13d09352a145ce

commit 848d7299a26988ea1bab3e576ba10762624dfff6
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Nov 20 09:04:28 2015 -0800

Ignore Content-Type from client on multipart-manifest=delete
    
    Otherwise, if a client includes something like:
    
        Content-Type: application/x-www-form-urlencoded
    
    ...we won't delete anything, and instead send back:
    
        Number Deleted: 0
        Number Not Found: 0
        Response Body:
        Response Status: 406 Not Acceptable
        Errors:
    
    ...despite the fact that we're managing the iterator, so we know it's
    acceptable.
    
    Change-Id: I791c7bda1d9df830d8dacd3783c2393db5a9ac09

commit c4eeea7820b7bbf6f5994c9878c8838c43e61b9b
Author: Sivasathurappan Radhakrishnan <siva.radhakrishnan@intel.com>
Date:   Thu Oct 29 23:04:42 2015 +0000

Added unit test cases for container/auditor.py
    
    Added unit test cases to cover all code path in _one_audit_pass
    function in container/auditor.py
    
    Change-Id: I8b89e94f1492e4366af3ac4260587e988ba29408

commit bdb7bf40321bc36b45a25742b6e1aed73576fabc
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Aug 27 01:09:41 2015 +0200

py3: Add py34 test environment to tox
    
    Add a py34 test environment to tox.ini using a whitelist
    of tests which pass on Python 3. Currently, only test_exceptions.py
    of test/unit/common/ is run on Python 3
    
    Change-Id: I5c8a9fe7b9635f1acab8f401908975fc6fc58c7a

commit 4dc4cbfc1ca50abc100d2302e928f9fc16f26195
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon Nov 16 14:27:51 2015 -0800

Fix 503 on zero-byte replicated PUT with incorrect Etag
    
    Closes-Bug: 1516579
    Change-Id: Iac91ed61254d3ca232521191fec25c19acb66413

commit 1b8b08039a3647ffdb30225b0939055982942159
Author: Zack M. Davis <zdavis@swiftstack.com>
Date:   Fri Oct 30 11:02:54 2015 -0700

remove remaining simplejson uses, prefer standard library import
    
    a1c32702, 736cf54a, and 38787d0f remove uses of `simplejson` from
    various parts of Swift in favor of the standard libary `json`
    module (introduced in Python 2.6). This commit performs the remaining
    `simplejson` to `json` replacements, removes two comments highlighting
    quirks of simplejson with respect to Unicode, and removes the references
    to it in setup documentation and requirements.txt.
    
    There were a lot of places where we were importing json from
    swift.common.utils, which is less intuitive than a direct `import json`,
    so that replacement is made as well.
    
    (And in two more tiny drive-bys, we add some pretty-indenting to an XML
    fragment and use `super` rather than naming a base class explicitly.)
    
    Change-Id: I769e88dda7f76ce15cf7ce930dc1874d24f9498a

commit e534af645d4ff051e97e552e98271810b5588cfe
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Nov 13 12:38:56 2015 -0800

Remove references to now-missing havana docs
    
    Change-Id: I3c708bacd468e923e574bdd5afc7acc58b072159

commit 7b7c6c5249ba598f10770308e05f3d2ad4e60871
Author: Lisak, Peter <peter.lisak@firma.seznam.cz>
Date:   Fri Oct 2 09:20:17 2015 +0200

swift-init return codes
    
    Currently, swift-init returns zero if can't locate config on start.
    Because of this problem, it is not possible to distinguish if managed
    to start server.
    
    Due to legacy two new complementary options are added. Default is context
    dependent.
    --strict returns non-zero if some config is missing (default mode
    if explicitly named server)
    --non-strict returns zero even if some config is missing (default mode
    if alias is used)
    
    As a side effect:
    If some of demanded servers already running it does not try to start
    unstarted and also returns non-zero (in strict mode). That is still sufficient
    for the goal of patch.
    
    For future improvements LSB status codes should be considered.
    
    DocImpact
    Change-Id: I7750abd4a94875b46f83f4aeee8509388d543c2b

commit 84891ba5a1a06013779c9be24f69b26fea0cb4f0
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Sun Nov 15 19:25:05 2015 +0100

New swauth URL
    
    Change-Id: I019f921417d2bd5325141e8a6873ee934fdf3f49

commit 001dc2531899a61480ab7819ebe8766dbf3d7d22
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Sat Nov 14 20:14:46 2015 +0100

Missing log_name option added
    
    Change-Id: If0224f346aa7372268115284c112e8f3a60a9be4

commit 376c20d1889ab764b4a501c59ba6539b051f04bb
Author: Zack M. Davis <zdavis@swiftstack.com>
Date:   Fri Nov 13 12:08:03 2015 -0800

remove Python 2.6-specific logging workaround
    
    0ff39c14 (December 2013) introduced this LoggingHandlerWeakRef to work
    around a problem in which file descriptors would be leaked under Python
    2.6. But we don't support 2.6 anymore.
    
    Change-Id: I1cd8249dfe17bce9cb301d13b761c11d2efd11d3

commit 6e95fb5d164e6fdc3bf036c40dd54a19fd67199a
Author: John Dickinson <me@not.mn>
Date:   Fri Nov 13 09:46:02 2015 -0800

remove pbr from requirements.txt. It's not a run-time dependency
    
    Change-Id: I46f498122fccb0830304acba7f99fee64badfe12

commit c03d53ab7737e208d9c436088e41f60fdab96b30
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Nov 4 12:25:19 2015 -0800

Close EC fragment iterators in the GreenThread that's consuming them
    
    Otherwise, we might try to close them while they're already running.
    This should prevent conditional SLO requests from returning 500 when
    they should have returned 304 or 412.
    
    Change-Id: I075a892f901c18ab5c178c9c4a2f367b76ae78e2
    Related-Change: I156d873c72c19623bcfbf39bf120c98800b3cada

commit fd30df6e6572791e89a5360cecbf4800f1f08bb5
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Nov 11 07:55:51 2015 -0800

Small cleanup for unit/proxy/controllers/test_obj
    
    Follow up for https://review.openstack.org/#/c/236007
    
    This fixes following minor items:
    
    - Fix a 'raise Exception class' syntax to 'raise Exception instance'
    - Use original eventlet.Timeout instead of swift.exceptions.Timeout
      imported from eventlet.Timeout
    - Change Timeout to initiate w/o args (1st arguments should be timeout
      second and we don't have to set None if we don't want to set the sec)
    - Add a message argument to some Exception instances
    
    Change-Id: Iab608cd8a1f4d3f5b4963c26b94ab0501837ffe1

commit 7f4139bc2669de07f55e8689956d87d9c78cc834
Author: Paul Dardeau <paul.dardeau@intel.com>
Date:   Wed Oct 28 22:25:24 2015 +0000

Added unit tests for ringbuilder command-line utility
    
    Added new unit tests:
    
    test_add_device_old_missing_region
    test_create_ring_number_of_arguments
    test_add_duplicate_devices
    test_rebalance_with_seed
    test_set_overload_number_of_arguments
    test_main_no_arguments
    test_main_single_argument
    test_main_with_safe
    
    Modified existing unit tests to create sample ring at start of test.
    This change was needed to have unit tests run correctly and demonstrate
    code coverage.
    
    test_unknown
    test_search_device_number_of_arguments
    test_list_parts_number_of_arguments
    test_set_weight_number_of_arguments
    test_set_info_number_of_arguments
    test_remove_device_number_of_arguments
    test_set_min_part_hours_number_of_arguments
    test_set_replicas_number_of_arguments
    test_set_replicas_invalid_value
    
    Updates to handled nested mocks.
    Updates to handle no exception case when SystemExit is expected.
    PEP8 corrections.
    
    Moved new tests from try blocks to use of assertRaises or call to
    run_srb using exp_results with specified exit codes.
    
    Updated run_srb to accept a dictionary of expected results. Specifically,
    look for 'valid_exit_codes' to test, default to (0,1).
    
    Change-Id: I4cf3f5f055a9babba140c68a9c7ff90b9c50ea62

commit ce173e9ed2e14692899a53da7eefe31f4a7c61d0
Author: Sivasathurappan Radhakrishnan <siva.radhakrishnan@intel.com>
Date:   Fri Nov 6 01:49:58 2015 +0000

Added unit test cases for server.py
    
    Added unit test cases to cover all code paths in REPLICATE and __call__
    functions in account/server.py and container/server.py
    
    Change-Id: Ia335e9a6668821d3e34b12fc3a133a707880e87f

commit 2d85a3f6997fbadac210207f841c4ec25ff50cc4
Author: Tushar Gohad <tushar.gohad@intel.com>
Date:   Fri Oct 23 13:16:33 2015 +0000

EC: Use best available ec_type in unittests
    
    To minimize external library dependencies for Swift unit
    tests and SAIO, PyECLib 1.1.1 introduces a native backend
    'liberasurecode_rs_vand.'  This patch is to migrate over
    the unit tests to the new ec_type when available.
    
    This change will work with current pyeclib requirements
    (==1.0.7) and also future requirements (>=1.0.7).
    
    When we're able to raise *our* requirements to >=1.1.1 we
    should remove jerasure from the list of preferred backends.
    Related SAIO doc and example config changes should be
    included with that patch.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Change-Id: Idf657f0acf0479bc8158972e568a29dbc08eaf3b

commit 66dc1eebb1db51c81891b1f7bcc3e85aef9b8c1d
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Fri Oct 16 11:27:34 2015 -0500

ObjectControllers return application errors as 499 on bad read
    
    In the _transfer_data method, we translate all (Exception, Timeout)
    into a 499 whereas we should consider translating them to 500 on
    particular returning error scenarios.
    
    This affects both ReplicatedObjectController and ECObjectControllear.
    
    Change-Id: I571bbc5b1451243907b094a5718c8735fd824268
    Closes-Bug: 1504299

commit 4d0d76eb2079885c55711238906ef4ddd252aa86
Author: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
Date:   Tue Nov 3 16:43:57 2015 +0000

Add unit tests for direct_client
    
    Add tests for direct_delete_account, direct_get_object, retry
    Tests to validate possible argument calls of direct_put_object
    Extend FakeConn to support chunks in response
    
    Change-Id: I00cbc6c535ed8b3fb29bc07fdc51fdbb1220ff10
    Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>