Bug #1409302 “Use of GreenAsyncPile can lose txn_id logging” : Bugs : OpenStack Object Storage (swift)

clayg (clay-gerrard) on 2015-07-20

Changed in swift:
status:	New → Confirmed
importance:	Undecided → Medium

Revision history for this message

clayg (clay-gerrard) wrote on 2015-10-08:

#1

I wonder if we could make a factory for GreenAsyncPile's on a base wsgi application class such that when you get a pile it has the property of always passing in it's app's logger's thread locals into a wapped version of your spawned callable that fixes the locals inside of the spawn before calling your function...

Or maybe a TransactionIDPreservingGreenAsyncPile class that takes a logger... that's probably simpler - we might even be able to audit for usage of GreenAsyncPile...

janonymous (janonymous) on 2016-01-07

Changed in swift:
assignee:	nobody → janonymous (janonymous)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-01-12: Fix proposed to swift (master)

#2

Fix proposed to branch: master
Review: https://review.openstack.org/266239

Changed in swift:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-02-12: Change abandoned on swift (master)

#3

Change abandoned by janonymous (<email address hidden>) on branch: master
Review: https://review.openstack.org/266239
Reason: I think i squashed the changes but parent changed, Please find the patch after squashing : https://review.openstack.org/#/c/278176/

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-02-17: Fix merged to swift (master)

#4

Reviewed: https://review.openstack.org/278176
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=4906b4c431edc436f165b163a228b0a221950c79
Submitter: Jenkins
Branch: master

commit 4906b4c431edc436f165b163a228b0a221950c79
Author: Janonymous <email address hidden>
Date: Tue Jan 12 12:50:43 2016 +0530

Fix missing txn_id logs in GreenAsyncPile's spawned functions

    This commit ensures that the logger thread_locals
    value is passed to and set in _get_conn_response methods
    executed in a green thread.

    Added partial bug tag because in bug description a more
    relevant fix is suggested which would fix the bug completely
    but for now this makes sense to add this commit for logging.

    Co-Authored-By: Clay Gerrard <email address hidden>
    Change-Id: I13bbf174fdca89318d69bb0674ed23dc9ec25b9a
    Partial-Bug: #1409302

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-02-18: Fix proposed to swift (feature/crypto)

#5

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/281742

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-02-18: Fix merged to swift (feature/crypto)

#6

Download full text (11.1 KiB)

Reviewed: https://review.openstack.org/281742
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=be911d8abf9122bf69064e82296348b62973ec2f
Submitter: Jenkins
Branch: feature/crypto

commit e46d6b17251ab538595b1deb5581360a286b8fd1
Author: Christian Schwede <email address hidden>
Date: Wed Feb 17 20:47:26 2016 +0100

Add note on using printable chars for swift_hash_path_suffix/prefix

    Using unprintable characters for swift_hash_path_prefix/suffix might lead to
    hard problems when parsing of these values changes, for example due to newer
    Python versions or changes in the parsers itself. Let's avoid this and add a
    note that deployers should use printable strings for these values.

Change-Id: I976982b753b6af831ab91d7190f50f8f15bf73bf

commit b8fa48080812ab0f6bac7ae19c0290da08f6a6e4
Author: Donagh McCabe <email address hidden>
Date: Thu Feb 4 16:19:13 2016 +0000

Added links to API reference and usage guides

    Added a link to the API reference (where headers and query
    strings are documented; click the "detail" button to see).
    Also added a reference to Swift section of the OpenStack end
    user guide. This contains some additional details about the API.

    No attempt was made to reconcile duplicate information. Instead
    this patch links documents that might otherwise be overlooked.
    However, I fixed text, originally in a table, that had become
    garbled in a prior patch.

Change-Id: I0910cbeb0c8bffc00e510f35585603e7b7a67790

commit aa7204d106ae33eba9219514f31f008510c9db53
Author: Kazuhiro MIYAHARA <email address hidden>
Date: Fri Feb 12 20:48:34 2016 +0900

Remove '#! /usr/bin/env python' from unexecutable files

    'cli/recon.py' and 'cli/ring_builder_analyzer.py' have '#! /usr/bin/env
    python' in spite of they don't have execute permissions. This patch
    removes '#! /usr/bin/env python' from them.

Change-Id: I1917ccc84b1673af3d862be1796f54595f94c5ca

commit 8eb30afd7c39f6e6ba031f72c6a61f45b28d37f6
Author: Tim Burke <email address hidden>
Date: Tue Feb 16 10:25:39 2016 -0800

Static methods should be @staticmethods

Change-Id: Ifee5d68e00bbb3571aaac885cdd7490c79732985

commit 42f4b3fc1e69a046c1e70e73afe847e4423fa2da
Author: Christian Schwede <email address hidden>
Date: Tue Feb 16 10:08:55 2016 +0000

Add SwiftHLM to associated projects

Change-Id: I5ea3de34e61f22abec803b35fec8adde18a793e9

commit 025ebf2901d5a972796b73538bb306847d7d06b6
Author: Ankur Gupta <email address hidden>
Date: Wed Feb 10 14:36:16 2016 -0600

Docstring omission in class BaseDiskFileManager.

Added documentation for missing Docstring variables.

Change-Id: I29a53b8141c5607815f234a6123e2289200bca34

commit 4906b4c431edc436f165b163a228b0a221950c79
Author: Janonymous <email address hidden>
Date: Tue Jan 12 12:50:43 2016 +0530

Fix missing txn_id logs in GreenAsyncPile's spawned functions

    This commit ensures that the logger thread_locals
    value is passed to and set in _get_conn_response methods
    executed in a green thread.

Added partial bug tag because in bu...

Reviewed:  https://review.openstack.org/281742
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=be911d8abf9122bf69064e82296348b62973ec2f
Submitter: Jenkins
Branch:    feature/crypto

commit e46d6b17251ab538595b1deb5581360a286b8fd1
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed Feb 17 20:47:26 2016 +0100

Add note on using printable chars for swift_hash_path_suffix/prefix
    
    Using unprintable characters for swift_hash_path_prefix/suffix might lead to
    hard problems when parsing of these values changes, for example due to newer
    Python versions or changes in the parsers itself. Let's avoid this and add a
    note that deployers should use printable strings for these values.
    
    Change-Id: I976982b753b6af831ab91d7190f50f8f15bf73bf

commit b8fa48080812ab0f6bac7ae19c0290da08f6a6e4
Author: Donagh McCabe <donagh.mccabe@hp.com>
Date:   Thu Feb 4 16:19:13 2016 +0000

Added links to API reference and usage guides
    
    Added a link to the API reference (where headers and query
    strings are documented; click the "detail" button to see).
    Also added a reference to Swift section of the OpenStack end
    user guide. This contains some additional details about the API.
    
    No attempt was made to reconcile duplicate information. Instead
    this patch links documents that might otherwise be overlooked.
    However, I fixed text, originally in a table, that had become
    garbled in a prior patch.
    
    Change-Id: I0910cbeb0c8bffc00e510f35585603e7b7a67790

commit aa7204d106ae33eba9219514f31f008510c9db53
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Fri Feb 12 20:48:34 2016 +0900

Remove '#! /usr/bin/env python' from unexecutable files
    
    'cli/recon.py' and 'cli/ring_builder_analyzer.py' have '#! /usr/bin/env
    python' in spite of they don't have execute permissions.  This patch
    removes '#! /usr/bin/env python' from them.
    
    Change-Id: I1917ccc84b1673af3d862be1796f54595f94c5ca

commit 8eb30afd7c39f6e6ba031f72c6a61f45b28d37f6
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Feb 16 10:25:39 2016 -0800

Static methods should be @staticmethods
    
    Change-Id: Ifee5d68e00bbb3571aaac885cdd7490c79732985

commit 42f4b3fc1e69a046c1e70e73afe847e4423fa2da
Author: Christian Schwede <cschwede@redhat.com>
Date:   Tue Feb 16 10:08:55 2016 +0000

Add SwiftHLM to associated projects
    
    Change-Id: I5ea3de34e61f22abec803b35fec8adde18a793e9

commit 025ebf2901d5a972796b73538bb306847d7d06b6
Author: Ankur Gupta <ankur.gupta@intel.com>
Date:   Wed Feb 10 14:36:16 2016 -0600

Docstring omission in class BaseDiskFileManager.
    
    Added documentation for missing Docstring variables.
    
    Change-Id: I29a53b8141c5607815f234a6123e2289200bca34

commit 4906b4c431edc436f165b163a228b0a221950c79
Author: Janonymous <janonymous.codevulture@gmail.com>
Date:   Tue Jan 12 12:50:43 2016 +0530

Fix missing txn_id logs in GreenAsyncPile's spawned functions
    
    This commit ensures that the logger thread_locals
    value is passed to and set in _get_conn_response methods
    executed in a green thread.
    
    Added partial bug tag because in bug description a more
    relevant fix is suggested which would fix the bug completely
    but for now this makes sense to add this commit for logging.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Change-Id: I13bbf174fdca89318d69bb0674ed23dc9ec25b9a
    Partial-Bug: #1409302

commit 511a1b8698889c430bf673ed916ae41a96b99c5c
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Fri Feb 12 19:01:59 2016 +0900

Remove execute permission from swift/cli/ringbuilder.py
    
    In Change-Id: Ia63f59a8faca1fad990784f27532ca07a2125454,
    'bin/swift-ring-builder' is renamed to 'swift/cli/ringbuilder.py'.
    However, execute permission and '#! /usr/bin/env python' are left on
    the file.
    
    This patch removes execute permission and '#! /usr/bin/env python'
    from the file.
    
    Change-Id: I66370f825ce9ed9a40ec21f2745aa6aff22c8709

commit 973d9358d2e6e029409d1902831b4680e02952c9
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Thu Feb 11 10:20:10 2016 +0100

Bump PyECLib requirement to >= 1.2.0
    
    This was discussed at last Swift meeting [1] and conclusion was
    to require newest PyECLib version.
    
    [1] http://eavesdrop.openstack.org/meetings/swift/2016/swift.2016-02-10-21.00.log.html#l-268
    
    Depends-On: I0fd58f26bc0a3e7715356ff74344981f22786152
    Change-Id: I54e89d99b48e00e614d2d457814c00354763c83b

commit d3dd137cad896c7f741592e88f076acab2636a3c
Author: Tomas Matlocha <tomas.matlocha@firma.seznam.cz>
Date:   Tue Jan 5 12:32:50 2016 +0100

Additional info log message for drive-audit
    
    An additional info log message was added for case of
    running drive-audit without failed device unmounting.
    
    Change-Id: I11abee40a712b6c6de65e63626b6f7f0a9c9f4c7

commit c4158bd674f34c8a0648c98974ba474fcd1c2b7a
Author: Sarafraj Singh <Sarafraj.Singh@intel.com>
Date:   Thu Feb 11 20:52:47 2016 +0000

Fixed params on autocreate_account docstring
    
    ... in proxy.controllers.base
    
    Change-Id: Ic5c27cbf0f19b65fcaa3e6587583f03e56d646a0

commit f56d18e143d0a38584e4c383ea9afe46e67a0af6
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Feb 11 10:59:43 2016 -0800

Fix typos in comments
    
    Change-Id: I4f98d447bd2ddd8eeb2f4da66d069bd7d6f00dc6

commit cc8dc88b55b44ca2a1fd65c049f3cfec22eb5525
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Thu Feb 4 16:02:42 2016 +0100

Script for checking sanity of manpages
    
    Change-Id: Ia27b0ecf6a3267eb51f8a6562721dca72f128915

commit bcada66b9065c94cd39498a5330c727cc575a6fe
Author: root <lr699s@att.com>
Date:   Wed Feb 10 09:21:53 2016 -0600

Removed unused parameter in server.py
    
    The variable max_large_object_get_time is no longer used and was
    removed to reflect the change.
    
    Change-Id: I43051181dcb38245de6d13fab63876e83f46fc39
    Closes-Bug: #1538834

commit b5311f63dbf5787bda3495a58c8ef7a6a557daa4
Author: gh159m <gh159m@att.com>
Date:   Wed Feb 10 10:36:59 2016 -0600

Removed default value for log_statsd_host
    
    Multiple files and documents showed that log_statsd_host had
    a default value, usually localhost.  This was incorrect, instead
    setting a value for log_statsd_host enables statsd logging.
    
    Removed any reference of log_statsd_host having a default value.
    
    Also changed descriptions to show setting a value enables logging.
    
    Change-Id: I3ca5c0e8b8e4981de3aa6db0c476072b5a59723d
    Closes-Bug: #1542227

commit b98c1c60f9c581c76677c19baae60ae720e3c7f3
Author: Christian Berendt <berendt@b1-systems.de>
Date:   Tue Feb 9 07:32:48 2016 +0100

Remove executable flag from sample config file
    
    swift.conf-sample should not be executbale
    
    Change-Id: I960058e2af10b43efc6324a7534d9a23c810e39a

commit 13edc9a865cd203ddf28a3f782426a1b3b94f7a1
Author: Sarafraj Singh <Sarafraj.Singh@intel.com>
Date:   Fri Feb 5 17:28:26 2016 +0000

Improved test coverage of form_signature
    
    Added four new test cases to execute path of invalid arguments. This
    improved test coverage of file specified from 80% to 91%.
    
    Change-Id: I63c2e7bab3f01121301d78b687687208a58401c0

commit c7e5e125003de1a0f288f79783e9a41650e6f686
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Fri Jan 29 14:22:19 2016 +0000

Enable in-process func tests to optionally use fast-post
    
    Running functional tests in the in-process mode uses
    the default value for proxy-server object_post_as_copy,
    which is True. This patch adds support for an environment
    variable to change this value to False so that the fast-post
    mode is tested when running in-process functional tests.
    
    The patch also adds a new tox environment func-in-process-fast-post
    which forces in-process functional testing with
    object_post_as_copy=False.
    
    The motivation for this change, apart from enabling configurable
    local testing, is to put support in place for an upstream CI job
    that will actually functionally test the fast-post mode, which is
    otherwise only covered by unit tests.
    
    There are currently two gate jobs that run the functional tests:
    
    - gate-swift-dsvm-functional runs the tests *twice* against a
      devstack swift service, once using tempauth and once using
      keystoneauth. The devstack swift service uses the default
      object_post_as_copy=True.
    
    - gate-swift-tox-func runs the func tests in in-process mode
      which also uses tempauth and object_post_as_copy=True. This
      duplicates one of the config scenarios above.
    
    With this change either the gate-swift-tox-func job or a new job
    could run the functional tests using object_post_as_copy=False.
    
    Change-Id: Ia37f6df1dc38e44ef7404fbf0a52f6fc22fae0c2

commit 01410129dac6903ce7f486997a48e36072fa0401
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Aug 21 18:15:25 2015 -0700

Make handoffs_first a more useful "mode"
    
    Based on experience using handoffs_first and feedback from other
    operators it has become clear that handoffs_first is only used during
    periods of problematic cluster behavior (e.g. full disks) when
    replication attempts are failing to quickly drain off the partitions
    from the nodes which they have been rebalanced from.
    
    In order to focus on the most important work (getting handoff partitions
    off the node) handoffs_first mode will abort the current replication
    sweep before attempting any primary suffix syncing if any of the handoff
    partitions were not removed for any reason - and start over with
    replication of handoffs jobs as the highest priority.
    
    Note that handoffs_first being enabled will emit a warning on start up,
    even if no handoff jobs fail, because of the negative impact it can have
    during normal operations by dog piling on a node that was temporarily
    unavailable.
    
    Change-Id: Ia324728d42c606e2f9e7d29b4ab5fcbff6e47aea

commit d9b22ac51cca579e97ec7bd5a6df3ad7bf56c3bf
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Oct 19 16:45:33 2015 +0200

Port parse_mime_headers() to Python 3
    
    Port swift.common.utils.parse_mime_headers() to Python 3:
    
    * On Python 3, tries to decode headers from UTF-8. If an header was
      was not encoded to UTF-8, decode the header from Latin1.
    * Update the parse_mime_headers() tests: on Python 3, HTTP header
      values are Unicode strings.
    
    This change is a follow-up of the change
    Ia5ee2ead67e36e8c6416183667f64ae255887736.
    
    Change-Id: I042dd13e9eb0e9844ccd832d538cdac84359ed42

commit 09b188f03c4a40838ea6b3e54e0714d93077b066
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Thu Jul 16 09:35:37 2015 +0900

Add functional test for access control (RBAC) with Keystone
    
    This patch adds test cases for PUT, DELETE, GET, HEAD, POST and OPTIONS
    requests to accounts, containers and objects using various combinations
    of users/projects, roles and/or service tokens.
    
    Change-Id: Iea8141ac74ad949a3ae7fa47fda3135d0f2612f6

tags:

added: in-feature-crypto

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-08: Fix proposed to swift (feature/hummingbird)

#7

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/290148

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-03-08: Fix merged to swift (feature/hummingbird)

#8

Download full text (71.7 KiB)

Reviewed: https://review.openstack.org/290148
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0f7f1de233919a0b046349a3e31ae7fc8675a1c5
Submitter: Jenkins
Branch: feature/hummingbird

commit d6b4587a554b51ba733b151e0d924735b63d07e0
Author: Olga Saprycheva <email address hidden>
Date: Tue Mar 8 10:57:56 2016 -0600

Removed redundant file for flake8 check

Change-Id: I4322978aa20ee731391f7709bbd79dee140fc703

commit 643dbce134140530eef2ae62c42fef1107f905ed
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Mar 8 06:35:49 2016 +0000

Imported Translations from Zanata

For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I96b8ff1287bf219c5f8d56a3a4868c1063a953f9

commit 83713d37f0331c5ce9d377f4b4e8724551ae30ca
Author: Daisuke Morita <email address hidden>
Date: Mon Mar 7 18:30:47 2016 -0800

Missing comments for storage policy parameter

There are missing comments about storege_policy_index so appropriate
comments are added.

Change-Id: I3de3f0e6864e65918ca1a13cce70f19c23d295f5

commit 2cff2dec3d1c4588f5103e39679c43b3dded6dcb
Author: Olga Saprycheva <email address hidden>
Date: Fri Mar 4 15:19:39 2016 -0600

Fixed pep8 and flake8 errors in doc/source/conf.py and updated flake8 commands in tox.ini to test it.

Change-Id: I2add370e4cfb55d1388e3a8b41f688a7f3f2c621

commit 043fbca6d08648baa314ea2236f1ccdca8785f16
Author: Christian Schwede <email address hidden>
Date: Fri Mar 4 09:33:17 2016 +0000

Remove Erasure Coding beta status from docs

    This removes notes stating support for Erasure coding as beta. Questions
    regarding the stability of EC are coming up regularly, and are often referring
    to the docs that state EC as still in beta.

Besides this, a note marking statsd support as beta has been removed as well.

Change-Id: If4fb6a5c4cb741d42953db3cee8cb17a1d774e15

commit 09c73b86e9255f28fbd4cf571a52c17d549a8f9a
Author: Pete Zaitcev <email address hidden>
Date: Thu Mar 3 10:24:28 2016 -0700

Fix a crash in exception printout

Says the number of arguments does not match the number of '%'.

Change-Id: I8b5e395a07328fb9d4ac7a19f8ed2ae1637bee3b

commit fad5fabe0a22e8a86635a66523dd3d3d3b1fa705
Author: Tim Burke <email address hidden>
Date: Thu Mar 3 15:07:08 2016 +0000

During functional tests, 404 response to a DELETE is successful

Previously, we would only consider 204 responses successful, which would
cause some spurious gate failures, such as

http://logs.openstack.org/66/287666/3/check/gate-swift-dsvm-functional/c6d2673/console.html#_2016-03-03_13_41_07_846

Change-Id: Ic8c300647924352a297a2781b50064f7657038b4

commit e91de49d6864b3794f8dc5acd9c1bf0c2f7409d1
Author: Alistair Coles <email address hidden>
Date: Mon Aug 10 10:30:10 2015 -0500

Update container on fast-POST

    This patch makes a number of changes to enable content-type
    metadata to be updated when using the fast-POST mode of
    operation, as proposed in the associated spec ...

Reviewed:  https://review.openstack.org/290148
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0f7f1de233919a0b046349a3e31ae7fc8675a1c5
Submitter: Jenkins
Branch:    feature/hummingbird

commit d6b4587a554b51ba733b151e0d924735b63d07e0
Author: Olga Saprycheva <osapryc@us.ibm.com>
Date:   Tue Mar 8 10:57:56 2016 -0600

Removed redundant file for flake8 check
    
    Change-Id: I4322978aa20ee731391f7709bbd79dee140fc703

commit 643dbce134140530eef2ae62c42fef1107f905ed
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Tue Mar 8 06:35:49 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I96b8ff1287bf219c5f8d56a3a4868c1063a953f9

commit 83713d37f0331c5ce9d377f4b4e8724551ae30ca
Author: Daisuke Morita <morita.daisuke@ntti3.com>
Date:   Mon Mar 7 18:30:47 2016 -0800

Missing comments for storage policy parameter
    
    There are missing comments about storege_policy_index so appropriate
    comments are added.
    
    Change-Id: I3de3f0e6864e65918ca1a13cce70f19c23d295f5

commit 2cff2dec3d1c4588f5103e39679c43b3dded6dcb
Author: Olga Saprycheva <osapryc@us.ibm.com>
Date:   Fri Mar 4 15:19:39 2016 -0600

Fixed pep8 and flake8 errors in doc/source/conf.py and updated flake8 commands in tox.ini to test it.
    
    Change-Id: I2add370e4cfb55d1388e3a8b41f688a7f3f2c621

commit 043fbca6d08648baa314ea2236f1ccdca8785f16
Author: Christian Schwede <cschwede@redhat.com>
Date:   Fri Mar 4 09:33:17 2016 +0000

Remove Erasure Coding beta status from docs
    
    This removes notes stating support for Erasure coding as beta. Questions
    regarding the stability of EC are coming up regularly, and are often referring
    to the docs that state EC as still in beta.
    
    Besides this, a note marking statsd support as beta has been removed as well.
    
    Change-Id: If4fb6a5c4cb741d42953db3cee8cb17a1d774e15

commit 09c73b86e9255f28fbd4cf571a52c17d549a8f9a
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Thu Mar 3 10:24:28 2016 -0700

Fix a crash in exception printout
    
    Says the number of arguments does not match the number of '%'.
    
    Change-Id: I8b5e395a07328fb9d4ac7a19f8ed2ae1637bee3b

commit fad5fabe0a22e8a86635a66523dd3d3d3b1fa705
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Mar 3 15:07:08 2016 +0000

During functional tests, 404 response to a DELETE is successful
    
    Previously, we would only consider 204 responses successful, which would
    cause some spurious gate failures, such as
    
    http://logs.openstack.org/66/287666/3/check/gate-swift-dsvm-functional/c6d2673/console.html#_2016-03-03_13_41_07_846
    
    Change-Id: Ic8c300647924352a297a2781b50064f7657038b4

commit e91de49d6864b3794f8dc5acd9c1bf0c2f7409d1
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Mon Aug 10 10:30:10 2015 -0500

Update container on fast-POST
    
    This patch makes a number of changes to enable content-type
    metadata to be updated when using the fast-POST mode of
    operation, as proposed in the associated spec [1].
    
    * the object server and diskfile are modified to allow
      content-type to be updated by a POST and the updated value
      to be stored in .meta files.
    
    * the object server accepts PUTs and DELETEs with older
      timestamps than existing .meta files. This is to be
      consistent with replication that will leave a later .meta
      file in place when replicating a .data file.
    
    * the diskfile interface is modified to provide accessor
      methods for the content-type and its timestamp.
    
    * the naming of .meta files is modified to encode two
      timestamps when the .meta file contains a content-type value
      that was set prior to the latest metadata update; this
      enables consistency to be achieved when rsync is used for
      replication.
    
    * ssync is modified to sync meta files when content-type
      differs between local and remote copies of objects.
    
    * the object server issues container updates when handling
      POST requests, notifying the container server of the current
      immutable metadata (etag, size, hash, swift_bytes),
      content-type with their respective timestamps, and the
      mutable metadata timestamp.
    
    * the container server maintains the most recently reported
      values for immutable metadata, content-type and mutable
      metadata, each with their respective timestamps, in a single
      db row.
    
    * new probe tests verify that replication achieves eventual
      consistency of containers and objects after discrete updates
      to content-type and mutable metadata, and that container-sync
      sync's objects after fast-post updates.
    
    [1] spec change-id: I60688efc3df692d3a39557114dca8c5490f7837e
    
    Change-Id: Ia597cd460bb5fd40aa92e886e3e18a7542603d01

commit 3c61ab4678a7aa9ff256ace4bc97ab449607fd49
Author: asettle <alexandra.settle@rackspace.com>
Date:   Wed Feb 10 17:58:05 2016 +1000

Operational procedures guide
    
    This is the operational procedures guide that HPE used
    to operate and monitor their public Swift systems.
    It has been made publicly available.
    
    Change-Id: Iefb484893056d28beb69265d99ba30c3c84add2b

commit 8e4347afd59aa30708ef03a78202652aaadda2e9
Author: Takashi Kajinami <kajinamit@nttdata.co.jp>
Date:   Tue Nov 17 16:15:59 2015 +0900

Fix proxy-server's support for chunked transferring in GET object
    
    Proxy-server now requires Content-Length in the response header
    when getting object and does not support chunked transferring with
    "Transfer-Encoding: chunked"
    
    This doesn't matter in normal swift, but prohibits us from putting
    any middelwares to execute something like streaming processing of
    objects, which can't calculate the length of their response body
    before they start to send their response.
    
    Change-Id: I60fc6c86338d734e39b7e5f1e48a2647995045ef

commit 9dda1c9526c5b3e0ac905985921605d833bef3cc
Author: Takashi Kajinami <kajinamit@nttdata.co.jp>
Date:   Fri Dec 25 10:30:01 2015 +0900

Update parameters about authtoken middleware in proxy-server.conf.sample
    
    This patch replaces some configration parameters about authtoken
    middleware in proxy-server.conf, which will be deprecated.
    
    The following parameters are to be deprecated.
     auth_admin_prefix, auth_host, auth_port, auth_protocol,
     identity_uri, admin_token, admin_user, admin_password,
     admin_tenant_name
    
    Change-Id: I373734933189c87c4094203b0752dd3762689034

commit 087fa4fa01fc36343f2ce0974d3b36ec0b98afc8
Author: Andreas Jaeger <aj@suse.com>
Date:   Mon Feb 29 20:00:38 2016 +0100

Remove unused pngmath Sphinx extension
    
    There's no RST file that uses ".. math" and thus
    the pngmath Sphinx extension is not used and can
    get removed.
    
    Change-Id: I3ea4b529025c8ac3c4092f3720124647861e6668

commit 9c5b01e7c7699fbfd221e98ccde1afa9361c8119
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Wed Dec 9 08:08:22 2015 +0000

Add container quota functional test
    
    This patch provides one container quota functional test.
    
    Change-Id: I16354c15fc434d70f9bb6b9b1e18b09357195c83

commit efdc6ee8dbc434e9189a6b85b924c322c3ff6ae3
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Thu Dec 10 01:15:30 2015 +0000

Add X-Delete-After and X-Delete-At functional test
    
    This patch provides functional tests for X-Delete-After and X-Delete-At.
    
    Change-Id: I55b0d0584428413aa7a31f66defec55f9b1bbcd9

commit bd93d44bb46350f716f1dfb66fb43c2ab4ad8c50
Author: Takashi Kajinami <kajinamit@nttdata.co.jp>
Date:   Sun Feb 28 13:41:19 2016 +0900

Make sure all temp files get deleted in test_ringbuilder
    
    This patch makes test_ringbuilder create a temporal directory,
    run ring builder commands under it, and delete it after testing
    for each test cases, to fix temp file leaking.
    
    Change-Id: I6f59fe095ea6485af0e60b5a8e8fc3892e0a0f90

commit 10b5765c60790f8a3ae2a94d6a51736ec2c99ce0
Author: benjkeller <bjkeller@us.ibm.com>
Date:   Thu Feb 25 18:54:00 2016 -0600

Rename test_object_delete_at_aysnc
    
    This renames test_object_delete_at_aysnc to
    test_object_delete_at_async to match with the function
    async_update which it tests.
    
    Change-Id: I726afcbf08a3449c7af2834e573b97be378a86cd
    Closes-Bug: 1550067

commit 9e53bb47efa28983109da8ceca5d316925bc6804
Author: John Dickinson <me@not.mn>
Date:   Wed Feb 24 09:36:40 2016 -0800

updated some weird whitespace character to a normal one
    
    Change-Id: I187fb8fc8d10f950bbebc586999eced0bc117432

commit 54e53ae87b6e87229dc77cb7c814ab9733ea7013
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Fri Feb 19 09:30:10 2016 +0000

Silence SkipTest noise in in-process test teardown
    
    setup_package and teardown_package get called twice when
    running functional tests using nose. With in-process mode
    the first call to teardown_package clears global config
    which causes the test swiftclient to raise a SkipTest error
    during the second call to teardown_package.
    
    To reproduce:
    SWIFT_TEST_IN_PROCESS=1 nosetests ./test/functional/test_account.py
    
    This patch simply tests for config existence before attempting
    to create a test swiftclient in the teardown_package function.
    
    Also fix a related issue whereby in_process flag would be reset to
    False during second call to setup_package, thus causing some of the
    in process cleanup in teardown_package to never be executed.
    
    Change-Id: I074dcd3d39aa46b262632024b047556ca471e8b8

commit 076134784b4eb1c2e55b28218dc0d64a57795d63
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Feb 23 17:28:08 2016 -0800

Add warn method back to LogAdapter
    
    The change in 684c4c04 to improve py3 support breaks older middleware
    that consumes the SwiftLogger.
    
    Good for Swift code to not to use warn because it's deprecated.
    
    Bad for LogAdapter to not faithfully provide the current python logging
    module API (deprecated or otherwise).
    
    Change-Id: Ib0baf66b7f33f91562867076ff0526a287d8e5a6

commit eb7ca115e6501edd9225dce8784b8b3d3e094bb1
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Feb 23 14:00:34 2016 -0800

Fix StatsD tests to not use real DNS
    
    In common/test_utils.py, TestStatsdLogging had the majority of its
    test cases calling the real socket.getaddrinfo(), which uses real
    DNS. This is very slightly slower than using a mock getaddrinfo() when
    the machine running the tests has functioning DNS, but on a machine
    with no network connection at all, the tests are excruciatingly slow
    due to timeouts.
    
    This commit mocks things out as appropriate. There's still one user of
    the real getaddrinfo(), but it's for ::1, so that's just local
    resolution based on /etc/hosts.
    
    Timing numbers for "./.unittests test.unit.common.test_utils:TestStatsdLogging":
    
     * network, without this patch: 1.8s
     * no network, without this patch: 221.2s (ouch)
     * network, with this patch: 1.1s
     * no network, with this patch: 1.1s
    
    Change-Id: I1a2d6f24fc9bb928894fb1fd8383516250e29e0c

commit b97adf70ade2982fea5354c55f1eea9f98442fa7
Author: Thiago da Silva <thiago@redhat.com>
Date:   Tue Feb 23 16:23:51 2016 -0500

fix formatting of bulk operations doc
    
    Change-Id: I247d2867c55d6bbf2288dfb74271f57c6f88b243

commit cfd50212358372694ba0713ff8c3bc313c6f1a92
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Feb 18 21:06:40 2016 -0800

dry out container unittests
    
    Change Ifcc4392ffa4df485ce3c62a35d0fcc8c93de74aa introduced
    TestContainerController4Replicas as a subclass of
    TestContainerController. And there is duplicated code
    block to override setUp method.
    
    This patch removed the duplicated code block.
    
    Change-Id: Iad0b64cba52554b89dbd959475a2ce7c0d9a20eb

commit 406e8996fb288e9851e02c3f3faa536a9919b28f
Author: Takashi Kajinami <kajinamit@nttdata.co.jp>
Date:   Mon Feb 22 16:10:09 2016 +0900

Use 201 insted of 200 in PUT test case
    
    There are some unit test cases which assumes 200 as the response
    of PUT request, but 200 is never returned in PUT.
    This patch replaces 200 in PUT by 201, and adapt unit test cases
    to real case.
    
    Change-Id: I35fc7a4fe7fcc5558a9e6c6371105793c2710f3e

commit 5b18455cef57d052df3cda3e9179c923d3aadcad
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Fri Jan 22 07:06:45 2016 +0000

Add unit tests for checking responses in ContainerController
    
    This patch provides unit tests to check responses for various situation
    in three/four replicas env.
    
    Change-Id: Ifcc4392ffa4df485ce3c62a35d0fcc8c93de74aa

commit 002513d21786df0d6c71e9772ec4688e2c14d8de
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jan 7 11:56:33 2016 -0800

Make write-only container access consistent
    
    Previously, if a user could write to (but not read from) a container,
    the behavior for object POST would vary depending on whether
    object_post_as_copy was enabled (403 response) or disabled (202
    response).
    
    Now, POSTs will consistently be allowed, regardless of whether fast-POST
    is enabled.
    
    Change-Id: I1d6dcbc4f5034a322a1073850fc3b059ebb1c0fa

commit 9a6c6b55713f896609b7127560ae99fabbf759e7
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Thu Feb 18 17:38:42 2016 +0100

Fixed log format in object auditor
    
    Change-Id: I218e08446c5ffda2db7c774ec8bb9d28ce9f63e7

commit 2d55960a221c9934680053873bf1355c4690bb19
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Jan 14 18:31:21 2016 +0000

Fix inconsistent suffix hashes after ssync of tombstone
    
    Consider two replicas of the same object whose ondisk files
    have diverged due to failures:
    
      A has t2.ts
      B has t1.data, t4.meta
    
    (The DELETE at t2 did not make it to B. The POST at t4 was
    rejected by A.)
    
    After ssync replication the two ondisk file sets will not be
    consistent:
    
      A has t2.ts  (ssync cannot POST t4.meta to this node)
      B has t2.ts, t4.meta (ssync should not delete t4.meta,
                            there may be a t3.data somewhere)
    
    Consequenty the two nodes will report different hashes for the
    object's suffix, and replication will repeat, always with the
    inconsistent outcome. This scenario is reproduced by the probe
    test added in this patch.
    
    (Note that rsync replication does result in (t2.ts, t4.meta)
    on both nodes.)
    
    The solution is to change the way that suffix hashes are
    calculated. Currently the names of *all* files found in each
    object dir are added to the hash.  With this patch the
    timestamps of only those files that could be used to
    construct a valid diskfile are added to the hash. File
    extensions are appended to the timestamp so that in most
    'normal' situations the result of the hashing is the same
    as before this patch. That avoids a storm of hash mismatches
    when this patch is deployed in an existing cluster.
    
    In the problem case described above, t4.meta is no longer
    added to the hash, since it is not useful for constructing
    a diskfile. (Note that t4.meta is not deleted because it
    may become useful should a t3.data be replicated in future).
    
    Closes-Bug: 1534276
    Change-Id: I99e88b8d5f5d9bc22b42112a99634ba942415e05

commit ddeb0cde9fc828fa79dce90df4975f66737ba054
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Feb 17 21:17:34 2016 -0800

Fix object-server to handle newer ts file
    
    Swift object-server will return 409 Conflict when the incomming
    request X-Timestamp is older than the local disk file to prevent
    wasted data transfer and disk space consumption. However, IFF,
    the local disk file is a tombstone, current object-server will
    write the data in the device and cleanup it according to timestamp
    comparison result in hash_cleanup_list_dir, and then return 201
    Created. That's wasted and far from the semantics in the normal case.
    
    This patch fixes object-server to catch the DiskFileDeleted exception
    at PUT request, and then, set the timestamp from tombstone as
    original_timestamp to be able to compare with request timestamp.
    
    Co-Authored-By: Kazuhiro Miyahara <miyahara.kazuhiro@lab.ntt.co.jp>
    Change-Id: I078c9cb90707a3c320708e76ea42cbfa73e1ea4b
    Closes-Bug: #1546865

commit 878688cae00576ec0bdb0a66eca7ada95e661c0a
Author: Béla Vancsics <vancsics@inf.u-szeged.hu>
Date:   Mon Dec 7 17:02:40 2015 +0100

Reduce code duplication
    
    Reduced source code by extracting duplicated code
    (swift/common/direct_client.py)
    http://openqa.sed.hu/dashboard/index/5?did=1
    
    Change-Id: I47fa6ae12c646e8e42fb0a46a017971f05d883fc
    Closes-Bug: 1536067

commit c3201f256cf415988a3cb1c42e9c6b7835195fa3
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Fri Jan 8 18:03:33 2016 +0900

Remove execute permissions from doc files and swift.conf-sample
    
    Some doc files and swift.conf-sample were added execute permissions in past changes.
    This patch remove execute permissions from them.
    
    Change-Id: Id8844989a8321578e9207566ebd6660f5b9523f0

commit e46d6b17251ab538595b1deb5581360a286b8fd1
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed Feb 17 20:47:26 2016 +0100

Add note on using printable chars for swift_hash_path_suffix/prefix
    
    Using unprintable characters for swift_hash_path_prefix/suffix might lead to
    hard problems when parsing of these values changes, for example due to newer
    Python versions or changes in the parsers itself. Let's avoid this and add a
    note that deployers should use printable strings for these values.
    
    Change-Id: I976982b753b6af831ab91d7190f50f8f15bf73bf

commit b8fa48080812ab0f6bac7ae19c0290da08f6a6e4
Author: Donagh McCabe <donagh.mccabe@hp.com>
Date:   Thu Feb 4 16:19:13 2016 +0000

Added links to API reference and usage guides
    
    Added a link to the API reference (where headers and query
    strings are documented; click the "detail" button to see).
    Also added a reference to Swift section of the OpenStack end
    user guide. This contains some additional details about the API.
    
    No attempt was made to reconcile duplicate information. Instead
    this patch links documents that might otherwise be overlooked.
    However, I fixed text, originally in a table, that had become
    garbled in a prior patch.
    
    Change-Id: I0910cbeb0c8bffc00e510f35585603e7b7a67790

commit aa7204d106ae33eba9219514f31f008510c9db53
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Fri Feb 12 20:48:34 2016 +0900

Remove '#! /usr/bin/env python' from unexecutable files
    
    'cli/recon.py' and 'cli/ring_builder_analyzer.py' have '#! /usr/bin/env
    python' in spite of they don't have execute permissions.  This patch
    removes '#! /usr/bin/env python' from them.
    
    Change-Id: I1917ccc84b1673af3d862be1796f54595f94c5ca

commit db87a5a9889d436f2e2aacdc8923d55ca0d3d1ca
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Feb 16 10:52:01 2016 -0800

Mock sleep in direct_client retry tests
    
    ... and fixup some assertions, and other mocks
    
    Change-Id: Iadf839d1d3d735109fed24049fc3d829776b7b65

commit 8eb30afd7c39f6e6ba031f72c6a61f45b28d37f6
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Feb 16 10:25:39 2016 -0800

Static methods should be @staticmethods
    
    Change-Id: Ifee5d68e00bbb3571aaac885cdd7490c79732985

commit 4a44e27e00795c5f12176ef346cd2249fca54fd8
Author: Chaozhe.Chen <chaozhe.chen@easystack.cn>
Date:   Tue Feb 16 23:44:27 2016 +0800

Replace assertEqual(None, *) with assertIsNone in tests
    
    As swift no longer supports Python 2.6, replace assertEqual(None, *)
    with assertIsNone in tests to have more clear messages in case of
    failure.
    
    Change-Id: I94af3e8156ef40465d4f7a2cb79fb99fc7bbda56
    Closes-Bug: #1280522