OpenStack Object Storage (swift)

Invalid data in hashes.pkl doesn't get fixed

Bug #1830881 reported by Christian Schwede on 2019-05-29

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Object Storage (swift)	Fix Released	Undecided	Christian Schwede

Bug Description

If the data in hashes.pkl is invalid (eg. corrupted, but pickle still able to load it) it will raise TypeErrors and gets never fixed. For example:

May 29 09:58:11 controller-0 object-server: Starting object replication pass.
May 29 09:58:11 controller-0 object-server: Error syncing partition: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/obj/replicator.py", line 407, in update#012 self.replication_cycle))#012 File "/usr/lib/python2.7/site-packages/swift/common/utils.py", line 3666, in tpool_reraise#012 raise resp#012TypeError: must be encoded string without NULL bytes, not unicode
May 29 09:58:11 controller-0 object-server: 1/1 (100.00%) partitions replicated in 0.01s (193.88/sec, 0s remaining)
May 29 09:58:11 controller-0 object-server: 0 successes, 0 failures
May 29 09:58:11 controller-0 object-server: Object replication complete. (0.00 minutes)
May 29 09:58:11 controller-0 object-server: Replication sleeping for 30 seconds.

May 29 09:58:24 controller-0 object-server: ERROR __call__ error with REPLICATE /d1/174 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/obj/server.py", line 1150, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/site-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, *args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/swift/obj/server.py", line 1123, in REPLICATE#012 device, partition, suffixes, policy)#012 File "/usr/lib/python2.7/site-packages/swift/obj/diskfile.py", line 1408, in get_hashes#012 self._get_hashes, device, partition, policy, recalculate=suffixes)#012 File "/usr/lib/python2.7/site-packages/swift/common/utils.py", line 3666, in tpool_reraise#012 raise resp#012TypeError: must be encoded string without NULL bytes, not unicode

A simple reproducer is to create a dict with a NULL byte, eg:

with open("/srv/node/d1/objects/174/hashes.pkl", "wb+") as fh:
fh.write(pickle.dumps({'\x00\x00\x00': False}))
fh.close()

This gets never fixed by the replicators.

Tags:

Christian Schwede (cschwede) on 2019-05-29

Changed in swift:
assignee:	nobody → Christian Schwede (cschwede)
status:	New → In Progress

Revision history for this message

Christian Schwede (cschwede) wrote on 2019-05-29:

Note: manual fix is to remove the hashes.pkl files from the partition directories that raise an error.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-05-29: Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.opendev.org/661930

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-06-17: Fix merged to swift (master)

Reviewed: https://review.opendev.org/661930
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=c9e78d15e1e05f23facf7c28758b442bb25bde68
Submitter: Zuul
Branch: master

commit c9e78d15e1e05f23facf7c28758b442bb25bde68
Author: Christian Schwede <email address hidden>
Date: Wed May 29 11:37:54 2019 +0200

Remove invalid dict entries from hashes.pkl

    If the data in a hashes.pkl is corrupted but still de-serialized without
    errors, it will mess up the replication and gets never fixed. This
    happens for example if one of the keys is a NULL byte.

This patch checks if the dict keys in hashes.pkl are valid strings and
invalidates it if not.

Closes-Bug: 1830881
Change-Id: I84b062d062ff49935feed0aee3e1963bb72eb5ea

Changed in swift:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-06-27: Fix proposed to swift (feature/losf)

Fix proposed to branch: feature/losf
Review: https://review.opendev.org/668007

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-07-11: Fix merged to swift (feature/losf)

Download full text (16.1 KiB)

Reviewed: https://review.opendev.org/668007
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=aca7474ca3c7bba54473fca35b6d88a3a1efdca4
Submitter: Zuul
Branch: feature/losf

commit 9d1b7497400e8c3a7159b11dd8455c55a31db985
Author: Tim Burke <email address hidden>
Date: Tue Mar 26 13:02:24 2019 -0700

py3: port staticweb and domain_remap func tests

    Drive-by: Tighten domain_remap assertions on listings, which required
    that we fix proxy pipeline placement. Add a note about it to the sample
    config.

Change-Id: I41835148051294088a2c0fb4ed4e7a7b61273e5f

commit 38a24571ad5c192bacaf60a5634ea66164dbbb71
Author: Tim Burke <email address hidden>
Date: Wed Jul 10 09:13:44 2019 -0700

functests: make container creation less flakey in test_object

Change-Id: If62d82beb202dea553776920a95c177518b162ab

commit 4c4bd778ea8fe8d02a2892524c7918da0ca25ea9
Author: Tim Burke <email address hidden>
Date: Wed Jul 3 09:52:41 2019 -0700

container-replicator: Add a timeout for get_shard_ranges

    Previously this had no timeout, which meant that the replicator might
    hang and fail to make progress indefinitely while trying to receive
    shard ranges.

    While we're at it, only call get_shard_ranges when the remote indicates
    that it has shard ranges for us to sync -- this reduces the number of
    requests necessary to bring unsharded replicas in sync.

Change-Id: I32f51f42d76db38271442a261600089404a00f91
Closes-Bug: #1835260

commit 345f577ff191ee01c3cb4626805338028815c2b4
Author: Tim Burke <email address hidden>
Date: Wed Jul 3 07:28:14 2019 -0700

s3token: fix conf option name

Related-Change: Ica740c28b47aa3f3b38dbfed4a7f5662ec46c2c4
Change-Id: I71f411a2e99fa8259b86f11ed29d1b816ff469cb

commit 76fde89261e1940daadb720c41df1a3595314a97
Author: Tim Burke <email address hidden>
Date: Mon Jun 17 09:25:52 2019 -0700

py3: Be able to read and write non-ASCII headers

    Apparently Python's stdlib got more picky about what a header should
    look like. As a result, if an account, container, or object had a
    non-ASCII metadata name (values were fine), the proxy-server wouldn't
    parse all of the headers. See https://bugs.python.org/issue37093 for
    more information.

    This presented several problems:
    - Since the non-ASCII header aborts parsing, we may lose important
      HTTP-level information like Content-Length or Transfer-Encoding.
    - Since the offending header wouldn't get parsed, the client wouldn't
      even know what the problem was.
    - Even if the client knew what the bad header was, it would have no way
      to clear it, as the server uses the same logic to parse incoming
      requests.

    So, hack in our own header parsing if we detect that parsing was
    aborted. Note that we also have to mangle bufferedhttp's putheader so we
    can get non-ASCII headers to the backend servers.

    Now, we can run the test_unicode_metadata tests in
    test/functional/test_account.py and test/functional/test_container.py
    under py2 against services running under py3.

Change-Id: I0f03c211f35a9a49e047a571...

Reviewed:  https://review.opendev.org/668007
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=aca7474ca3c7bba54473fca35b6d88a3a1efdca4
Submitter: Zuul
Branch:    feature/losf

commit 9d1b7497400e8c3a7159b11dd8455c55a31db985
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Mar 26 13:02:24 2019 -0700

py3: port staticweb and domain_remap func tests
    
    Drive-by: Tighten domain_remap assertions on listings, which required
    that we fix proxy pipeline placement. Add a note about it to the sample
    config.
    
    Change-Id: I41835148051294088a2c0fb4ed4e7a7b61273e5f

commit 38a24571ad5c192bacaf60a5634ea66164dbbb71
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jul 10 09:13:44 2019 -0700

functests: make container creation less flakey in test_object
    
    Change-Id: If62d82beb202dea553776920a95c177518b162ab

commit 4c4bd778ea8fe8d02a2892524c7918da0ca25ea9
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jul 3 09:52:41 2019 -0700

container-replicator: Add a timeout for get_shard_ranges
    
    Previously this had no timeout, which meant that the replicator might
    hang and fail to make progress indefinitely while trying to receive
    shard ranges.
    
    While we're at it, only call get_shard_ranges when the remote indicates
    that it has shard ranges for us to sync -- this reduces the number of
    requests necessary to bring unsharded replicas in sync.
    
    Change-Id: I32f51f42d76db38271442a261600089404a00f91
    Closes-Bug: #1835260

commit 345f577ff191ee01c3cb4626805338028815c2b4
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jul 3 07:28:14 2019 -0700

s3token: fix conf option name
    
    Related-Change: Ica740c28b47aa3f3b38dbfed4a7f5662ec46c2c4
    Change-Id: I71f411a2e99fa8259b86f11ed29d1b816ff469cb

commit 76fde89261e1940daadb720c41df1a3595314a97
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jun 17 09:25:52 2019 -0700

py3: Be able to read and write non-ASCII headers
    
    Apparently Python's stdlib got more picky about what a header should
    look like. As a result, if an account, container, or object had a
    non-ASCII metadata name (values were fine), the proxy-server wouldn't
    parse all of the headers. See https://bugs.python.org/issue37093 for
    more information.
    
    This presented several problems:
    - Since the non-ASCII header aborts parsing, we may lose important
      HTTP-level information like Content-Length or Transfer-Encoding.
    - Since the offending header wouldn't get parsed, the client wouldn't
      even know what the problem was.
    - Even if the client knew what the bad header was, it would have no way
      to clear it, as the server uses the same logic to parse incoming
      requests.
    
    So, hack in our own header parsing if we detect that parsing was
    aborted. Note that we also have to mangle bufferedhttp's putheader so we
    can get non-ASCII headers to the backend servers.
    
    Now, we can run the test_unicode_metadata tests in
    test/functional/test_account.py and test/functional/test_container.py
    under py2 against services running under py3.
    
    Change-Id: I0f03c211f35a9a49e047a5718a9907b515ca88d7

commit 4f7c44a9d7921849da0f3eecd23c26b65ec71435
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Jul 2 11:21:33 2019 -0700

Add information about secret_cache_duration to sample config
    
    Related-Change-Id: Id0c01da6aa6ca804c8f49a307b5171b87ec92228
    Change-Id: Ica740c28b47aa3f3b38dbfed4a7f5662ec46c2c4

commit 072b4045130c31dcc6f31f9618f3647a9c735a5f
Author: Kota Tsuyuzaki <kota.tsuyuzaki.pc@hco.ntt.co.jp>
Date:   Mon Jul 1 17:43:02 2019 +0900

s3api: add unit test case for s3acl
    
    To clarify the corner case when Swift is over-loaded, or user-errors.
    
    Change-Id: I41867290a63b329781dc7030f9f3ce0118e2032b

commit c46b88ab742761c8918013fa9cc40a169e015474
Author: zhufl <zhu.fanglei@zte.com.cn>
Date:   Fri Jun 28 16:54:27 2019 +0800

Fix invalid assert states
    
    "self.assertTrue(policies[1].is_deprecated, True)" and
    "self.assertTrue(crashy_calls[0], 1)" are not correct, this is
    to fix them.
    
    Change-Id: I7b07f0833d675d2939c910f679b54da2b8cda482

commit 88cec2c3966080fa79e5e623c7603c0eed4b94e3
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 27 12:29:04 2019 -0700

s3api: Put more information in the str() for InternalError
    
    There are few things more frustrating when digging through a log than
    finding a traceback that ends with
    
        raise InternalError('unexpected status code %d' % status)
    
    while having no indication of what the status code was.
    
    Change-Id: I08e9f7c0643ae0469bd1937a30c1c4477b67c31f

commit e02b9b798007b9f8c70e33c22fa6189d81a41ca3
Author: Yuxin Wang <wang.yuxin@ostorage.com.cn>
Date:   Thu Jul 5 17:46:50 2018 +0800

Use get_container_info to check existence before container PUT
    
    PUT request on an existing container will trigger an update on
    container db. When disks where container db landed are under
    heavy loads, update on the container db may fail due to LockTimout.
    
    Hence, we first check existence, if it's not there, we PUT.
    
    Change-Id: Ic61153948e35f1c09b05bfc97dfac3fb487b0898
    Closes-Bug: 1780204

commit 044c919871f80e1df91ff9a40318a5afd6a2960d
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Jun 25 13:42:26 2019 -0500

More tests for 404 handoff skipping
    
    Sometimes we want 404, sometimes we want 503 - it's tricky
    
    Change-Id: I30f5af07e2e1fc7cbb6bdb1c334a0a161caf0906
    Related-Change-Id: I53ed04b5de20c261ddd79c98c629580472e09961

commit 563e1671cf0583767d652a72ebc4d73d7cc07435
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Jun 25 11:53:32 2019 -0500

Return 503 when primary containers can't respond
    
    Closes-Bug: #1833612
    
    Change-Id: I53ed04b5de20c261ddd79c98c629580472e09961

commit 82169ead1cc1c60318507fc3f0fbd7deff0f15e9
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jun 24 16:55:59 2019 -0500

Don't handle object without container
    
    Closes-Bug: #1833616
    
    Change-Id: I16ca1589abcea2bca942da7e97719286a8961ea6

commit 55b04d16c4740fb731a2ffa97a84e1607ca94ef3
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jun 21 22:34:07 2019 -0700

Run all func tests under py2 against services running py3
    
    ... at least, on the py3 dsvm job.
    
    Depends-On: I0f03c211f35a9a49e047a5718a9907b515ca88d7
    Change-Id: Ia783ae10344380eb5afd9d43273969b433962945

commit 39a54fecdc5d7f9a09f34fcd9e81ec60948b51d7
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Apr 17 13:11:33 2019 -0700

py3: add swift-dsvm-functional-py3 job
    
    Note that keystone wants to stick some UTF-8 encoded bytes into
    memcached, but we want to store it as JSON... or something?
    
    Also, make sure we can hit memcache for containers with invalid UTF-8.
    Although maybe it'd be better to catch that before we ever try memcache?
    
    Change-Id: I1fbe133c8ec73ef6644ecfcbb1931ddef94e0400

commit ff4459ed6bcaa26bc4df9873577ccad9578a89fc
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jun 14 16:20:03 2019 -0700

Move call to global_conf_callback after loadapp()
    
    Otherwise, paste complains about not being able to interpolate values
    into the replication_semaphore. As long as it gets dropped in before we
    fork(), I think it's OK?
    
    Closes-Bug: 1691075
    Change-Id: Ib7e065c47871876786bcc9ff39737f5d1bb3c12c

commit 9f1ef35630ebf9da53cbd061f89889e534803725
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Tue Apr 16 16:59:51 2019 +1000

py3: Port the tempurl middleware
    
    This patch ports the tempurl middleware over to PY3. We use
    an "all-native" string model, where we convert WSGI strings
    as soon as we see them. It helps to deal with HMAC.
    
    Aaand, we fix formpost along the way. It _clearly_ was
    doing a wrong thing, encoding the same body several times
    if we had several keys. On py2 it wasn't noticeable, but
    on py3 this breaks, because the bytes type cannot be encoded
    again.
    
    Change-Id: I69974cc8a39731c980b54137b799a36b2e63a44a

commit a57085edc9427e3ee7c6aa474f2954f0d8825df3
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Wed Jun 12 19:42:44 2019 -0500

py3: port the test of locale
    
    We needed to change the sample translation a little, because
    Python 3 requires encoding metadata.
    
    We also remove workarounds for a specific bug in the subprocess
    module, which were needed on Python 2.6 only.
    
    Change-Id: Ie7791297cab16a649880088fa9eae6b4f95855f7

commit 4ab3706b75362b39cebf7ae5f325c191459c2690
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Tue Jun 18 14:47:39 2019 +0100

Remove nobarrier option
    
    XFS no longer supports nobarrier mount option.
    It has been deprecated for a long time[1] and removed in
    recent kernel versions resulting in an error when trying to
    mount: "kernel: XFS (loop0): unknown mount option [nobarrier]."
    
    [1] - https://patchwork.kernel.org/patch/9486549/
    
    Change-Id: Iaa9208fb20545ae9ac990f0e180899108d983123

commit 876ce231d616530e036521ba7cac28373e722ce2
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jun 17 14:41:19 2019 -0700

py3: increase _MAXHEADERS for eventlet, too
    
    eventlet.wsgi goes straight to eventlet.green.* instead of stdlib.
    
    Change-Id: Ie1f8808c04f975ccc7cea532c4a566d6984f1757

commit 6374734d7d0f76d863afb167637d0b590316e306
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri May 24 08:33:03 2019 -0700

Stop manually installing eventlet for our py3 tests
    
    0.25.0 is now included in upper-constraints.txt
    
    Related-Change: I04be480829eaf67ce328b2e7f7d7e2f70aaed90e
    Change-Id: I5b8ecd82d6915d0e1cb216c423bbcc75c50a6f3d

commit 014358018e34639da09ff7d28f8b095e3f7b882c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jun 17 11:24:59 2019 -0700

py3: Decode some things in s3api
    
    Query strings and upload IDs should be native strings.
    
    With this, you should be able to run the s3api functional tests under
    py2 against py3 services.
    
    Change-Id: I896c84872a5d63b23d6edabeb6d48ce7bc9a533e

commit ef664700d42db5bd2dd8be073598eee36ab1b258
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jun 14 15:41:13 2019 -0700

py3: Specify an encoding when loading db.pending pickles
    
    Otherwise you can hit UnicodeDecodeErrors when upgrading from py2 to
    py3.
    
    Change-Id: Iab808b3d0c2051badda0450015077ea8b1e5adb8

commit e6c971b7e0fe5ea9fcf905482dd0243f53013b0d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jun 14 14:38:33 2019 -0700

Stop sending non-UTF-8 prefixes in func tests
    
    Previously, we'd (with high likelihood) cut it off midway through
    a codepoint.
    
    Change-Id: Iba4cce213dce204d7d1b53a937bfbe537276f0f3

commit ccd3b890cdddca132b4d9cdd9372a9927bfb5aac
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Fri Jun 14 15:16:45 2019 +0100

Enable s3api in saio docker container
    
    Change-Id: Ie759c4092408a210d32f6457142457087343d33a

commit ef8818a6399e295ebf0dc891ef483e4840a61475
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed May 29 18:14:17 2019 -0700

Fix up how we memcache on py3
    
    Previously, we stored the WSGI strings in memcached and returned them when
    responding to get_account/container_info calls. This would lead to cache
    corruption in a heterogenous py2/py3 cluster such as you would have during
    a rolling upgrade.
    
    Now, only store and return native strings.
    
    Change-Id: I8d6f66dfe846493972e433f70bad76a33d204562

commit 0f46ace8a95c6abddba0cc5b02504a5324f1586c
Author: zhufl <zhu.fanglei@zte.com.cn>
Date:   Fri Jun 14 11:23:09 2019 +0800

Fix :param: in docstring
    
    In docstring :param should be used instead of :param:
    
    Change-Id: I55cd3fc188dca4a357dbb599aa58ab4083d3baf5

commit 668e45af6dbb548682c46aaf9d4c515b22bc3ea7
Author: Kota Tsuyuzaki <kota.tsuyuzaki.pc@hco.ntt.co.jp>
Date:   Mon Jun 10 15:27:49 2019 +0900

Update mailmap for Kota Tsuyuzaki
    
    His company changes the e-mail domain and the old one has been
    recieve-only so to update the author's e-mail address, add the
    entry to the mailmap.
    
    Change-Id: I0c1f77425d9cefb2e6340ffb4c65cdea956c216f

commit e8e7106d14f9bfe16427b5be300368c24c162481
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Apr 15 21:44:18 2019 -0700

py3: port obj/reconstructor tests
    
    All of the swift changes we needed for this were already done elsewhere.
    
    Change-Id: Ib2c26fdf7bd36ed1cccd5dbd1fa208f912f4d8d5

commit a122b720e9c61b38cfe502dcb63a9c119220c4c1
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jun 3 08:57:23 2019 -0700

Dockerfile cleanup
    
    - Use `set -e` in all install scripts. Clean up swift_needs to still
      work properly and exit cleanly under `set -e`.
    - Drop some redundant slashes.
    
    Change-Id: I05f38c6366e3da0aeed2b6451a8bd9d7dc531119

commit 17fd725707c9a36fc4536bd3c955042931e314b3
Author: Ehud Kaldor <ehud@unfairfunction.org>
Date:   Wed Jun 5 11:29:28 2019 -0700

Installing liberasurecode from Alpine Linux repos instead of building from source.
    
    closes #1831136
    
    Change-Id: Ic9a2844fffc6dba6a4d44a9712edbb5cd91f0470

commit 2eb2451685fdd2efa3f49bc48211844a35300ebf
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Apr 12 22:13:41 2019 -0700

Increase some middleware test coverage
    
    This patch increases the test coverage of the following middlewares:
     - list_endpoints
     - crypto
     - crossdomain
    
    Change-Id: I3dec85f61da07bd110bf42220d5ba46e11833a90

commit c9e78d15e1e05f23facf7c28758b442bb25bde68
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed May 29 11:37:54 2019 +0200

Remove invalid dict entries from hashes.pkl
    
    If the data in a hashes.pkl is corrupted but still de-serialized without
    errors, it will mess up the replication and gets never fixed. This
    happens for example if one of the keys is a NULL byte.
    
    This patch checks if the dict keys in hashes.pkl are valid strings and
    invalidates it if not.
    
    Closes-Bug: 1830881
    Change-Id: I84b062d062ff49935feed0aee3e1963bb72eb5ea

commit e28bc3c292a3c6fba67ba5f4c92068686994184f
Author: Mandell Degerness <mdegerness@swiftstack.com>
Date:   Wed May 22 16:28:36 2019 -0700

Fix incorrect setting of symlink_target_account
    
    The value of the symlink_target_account in etag override. It was previously
    being set to the object value, rather than the account value.
    
    Change-Id: I51889d7b19370ceaa66168103cb31d27a8ee3a4a
    Closes-Bug: #1829085

commit e22960fd718c4594bc8a504a12aba3f6a52175bf
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon May 20 21:54:40 2019 -0700

s3api: Fix ETag when copying a MU part from another MU
    
    Previously, we'd preserve the sysmeta that we wrote down with the
    original multipart-upload to track its S3-style etag on the new part,
    causing it to have an ETag like `<MD5>-<N>`. Later, when the client
    tried to complete the new multipart-upload, it would send that etag back
    to the server, which would reject the request because the ETag didn't
    look like a normal MD5.
    
    Now, have s3api include blank values in the copy request to overwrite
    the source sysmeta, and treat a blank etag override the same as a
    missing one.
    
    Change-Id: Id33a7ab9d0b8f33fede73eae540d6137708e1218
    Closes-Bug: #1829959

commit 53f8ccb4a3f5007db673a52ce17f80c50720fd49
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue May 21 14:03:47 2019 -0700

container: Unify storage-policy error paths
    
    While we're at it, use the right header name.
    
    Change-Id: I2238ec7fb0325167c53757b62ff9eb2ce5576fa4

commit 350bc8cbee515e9bb01e57c991851692e7fe0dd1
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon May 20 14:09:56 2019 -0700

s3api: stop adding etag quotes when there are already etag quotes
    
    Some middleware, such as that included with ProxyFS, may put
    RFC-compliant ETags in container listings; we should leave those
    unquoted, so we don't double-up on quotes.
    
    Change-Id: I8b56231aef41076a224dc62cd93c1973355c4670

tags:

added: in-feature-losf

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-07-22: Fix included in openstack/swift 2.22.0

This issue was fixed in the openstack/swift 2.22.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-08-08: Fix proposed to swift (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/675434

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-08-17: Fix merged to swift (stable/stein)

Reviewed: https://review.opendev.org/675434
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=46da6c1c01bc8beaf9a2b0a46122376b1bc7441f
Submitter: Zuul
Branch: stable/stein

commit 46da6c1c01bc8beaf9a2b0a46122376b1bc7441f
Author: Christian Schwede <email address hidden>
Date: Wed May 29 11:37:54 2019 +0200

Remove invalid dict entries from hashes.pkl

This patch checks if the dict keys in hashes.pkl are valid strings and
invalidates it if not.

    Closes-Bug: 1830881
    Change-Id: I84b062d062ff49935feed0aee3e1963bb72eb5ea
    (cherry picked from commit c9e78d15e1e05f23facf7c28758b442bb25bde68)

tags:

added: in-stable-stein

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-23: Fix proposed to swift (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/690737

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-11-08: Fix merged to swift (stable/rocky)

#10

Reviewed: https://review.opendev.org/690737
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=b90c87158df46513489f64e34224de7424859874
Submitter: Zuul
Branch: stable/rocky

commit b90c87158df46513489f64e34224de7424859874
Author: Christian Schwede <email address hidden>
Date: Wed May 29 11:37:54 2019 +0200

Remove invalid dict entries from hashes.pkl

This patch checks if the dict keys in hashes.pkl are valid strings and
invalidates it if not.

    Closes-Bug: 1830881
    Change-Id: I84b062d062ff49935feed0aee3e1963bb72eb5ea
    (cherry picked from commit c9e78d15e1e05f23facf7c28758b442bb25bde68)
    (cherry picked from commit 46da6c1c01bc8beaf9a2b0a46122376b1bc7441f)