Bug #1843313 “Sharding handoffs creates a *ton* of container-ser...” : Bugs : OpenStack Object Storage (swift)

Revision history for this message

Matthew Oliver (matt-0) wrote on 2019-09-10:

#1

cleaving_context_last_modified.patch Edit (4.5 KiB, text/plain)

Maybe something like the attached patch.

I'll play with it and add some tests and push it up for review in the next day or so.

Revision history for this message

Matthew Oliver (matt-0) wrote on 2019-09-17:

#2

I have pushed a patch for this: https://review.opendev.org/#/c/681970/

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-09-20: Fix merged to swift (master)

#3

Reviewed: https://review.opendev.org/681970
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=81a41da5420313f9cdb9c759bbb0f46c0d20c5af
Submitter: Zuul
Branch: master

commit 81a41da5420313f9cdb9c759bbb0f46c0d20c5af
Author: Matthew Oliver <email address hidden>
Date: Fri Sep 13 16:16:06 2019 +1000

Sharding: Clean up old CleaveConext's during audit

    There is a sharding edge case where more CleaveContext are generated and
    stored in the sharding container DB. If this number get's high enough,
    like in the linked bug. If enough CleaveContects build up in the DB then
    this can lead to the 503's when attempting to list the container due to
    all the `X-Container-Sysmeta-Shard-Context-*` headers.

    This patch resolves this by tracking the a CleaveContext's last
    modified. And during the sharding audit, any context's that hasn't been
    touched after reclaim_age are deleted.

This plus the skip empty ranges patches should improve these handoff
shards.

Change-Id: I1e502c328be16fca5f1cca2186b27a0545fecc16
Closes-Bug: #1843313

Changed in swift:
status:	New → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-03: Fix included in openstack/swift 2.23.0

#4

This issue was fixed in the openstack/swift 2.23.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-05: Fix proposed to swift (feature/losf)

#5

Fix proposed to branch: feature/losf
Review: https://review.opendev.org/686864

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-05: Fix merged to swift (feature/losf)

#6

Download full text (26.4 KiB)

Reviewed: https://review.opendev.org/686864
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=bfa8e9feb51f2b10adfec3a741661a76fcf73216
Submitter: Zuul
Branch: feature/losf

commit cb76e00e90aea834c8f3dd8a6ca5131acd43663b
Author: OpenStack Proposal Bot <email address hidden>
Date: Fri Oct 4 07:05:07 2019 +0000

Imported Translations from Zanata

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I40ce1d36f1c207a0d3e99a3a84a162b21b3c57cf

commit 527a57ffcdefc03a5080b07d63f0ded319e08dfe
Author: OpenStack Release Bot <email address hidden>
Date: Thu Oct 3 16:35:36 2019 +0000

Update master for stable/train

Add file to the reno documentation build to show release notes for
stable/train.

    Use pbr instruction to increment the minor version number
    automatically so that master versions are higher than the versions on
    stable/train.

Change-Id: Ia93e0b690f47c6231423a25dfd6a108a60378a21
Sem-Ver: feature

commit 8a4becb12fbe3d4988ddee73536673d6f55682dd
Author: Tim Burke <email address hidden>
Date: Fri Sep 27 15:18:59 2019 -0700

Authors/changelog for 2.23.0

Also, make some CHANGELOG formatting more consistent.

Change-Id: I380ee50e075a8676590e755f24a3fd7a7a331029

commit bf9346d88de2aeb06da3b2cde62ffa6200936367
Author: Tim Burke <email address hidden>
Date: Thu Aug 15 14:33:06 2019 -0700

Fix some request-smuggling vectors on py3

    A Python 3 bug causes us to abort header parsing in some cases. We
    mostly worked around that in the related change, but that was *after*
    eventlet used the parsed headers to determine things like message
    framing. As a result, a client sending a malformed request (for example,
    sending both Content-Length *and* Transfer-Encoding: chunked headers)
    might have that request parsed properly and authorized by a proxy-server
    running Python 2, but the proxy-to-backend request could get misparsed
    if the backend is running Python 3. As a result, the single client
    request could be interpretted as multiple requests by an object server,
    only the first of which was properly authorized at the proxy.

    Now, after we find and parse additional headers that weren't parsed by
    Python, fix up eventlet's wsgi.input to reflect the message framing we
    expect given the complete set of headers. As an added precaution, if the
    client included Transfer-Encoding: chunked *and* a Content-Length,
    ensure that the Content-Length is not forwarded to the backend.

    Change-Id: I70c125df70b2a703de44662adc66f740cc79c7a9
    Related-Change: I0f03c211f35a9a49e047a5718a9907b515ca88d7
    Closes-Bug: 1840507

commit 0217b12b6d7d6f3727a54db65614ff1ef52d6286
Author: Matthew Oliver <email address hidden>
Date: Wed Sep 4 14:30:33 2019 +1000

PDF Documentation Build tox target

This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:

https://governance.openstack.org/tc/goals/selected/train/pdf-doc-...

Reviewed:  https://review.opendev.org/686864
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=bfa8e9feb51f2b10adfec3a741661a76fcf73216
Submitter: Zuul
Branch:    feature/losf

commit cb76e00e90aea834c8f3dd8a6ca5131acd43663b
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Fri Oct 4 07:05:07 2019 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://docs.openstack.org/i18n/latest/reviewing-translation-import.html
    
    Change-Id: I40ce1d36f1c207a0d3e99a3a84a162b21b3c57cf

commit 527a57ffcdefc03a5080b07d63f0ded319e08dfe
Author: OpenStack Release Bot <infra-root@openstack.org>
Date:   Thu Oct 3 16:35:36 2019 +0000

Update master for stable/train
    
    Add file to the reno documentation build to show release notes for
    stable/train.
    
    Use pbr instruction to increment the minor version number
    automatically so that master versions are higher than the versions on
    stable/train.
    
    Change-Id: Ia93e0b690f47c6231423a25dfd6a108a60378a21
    Sem-Ver: feature

commit 8a4becb12fbe3d4988ddee73536673d6f55682dd
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Sep 27 15:18:59 2019 -0700

Authors/changelog for 2.23.0
    
    Also, make some CHANGELOG formatting more consistent.
    
    Change-Id: I380ee50e075a8676590e755f24a3fd7a7a331029

commit bf9346d88de2aeb06da3b2cde62ffa6200936367
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Aug 15 14:33:06 2019 -0700

Fix some request-smuggling vectors on py3
    
    A Python 3 bug causes us to abort header parsing in some cases. We
    mostly worked around that in the related change, but that was *after*
    eventlet used the parsed headers to determine things like message
    framing. As a result, a client sending a malformed request (for example,
    sending both Content-Length *and* Transfer-Encoding: chunked headers)
    might have that request parsed properly and authorized by a proxy-server
    running Python 2, but the proxy-to-backend request could get misparsed
    if the backend is running Python 3. As a result, the single client
    request could be interpretted as multiple requests by an object server,
    only the first of which was properly authorized at the proxy.
    
    Now, after we find and parse additional headers that weren't parsed by
    Python, fix up eventlet's wsgi.input to reflect the message framing we
    expect given the complete set of headers. As an added precaution, if the
    client included Transfer-Encoding: chunked *and* a Content-Length,
    ensure that the Content-Length is not forwarded to the backend.
    
    Change-Id: I70c125df70b2a703de44662adc66f740cc79c7a9
    Related-Change: I0f03c211f35a9a49e047a5718a9907b515ca88d7
    Closes-Bug: 1840507

commit 0217b12b6d7d6f3727a54db65614ff1ef52d6286
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Wed Sep 4 14:30:33 2019 +1000

PDF Documentation Build tox target
    
    This patch adds a `pdf-docs` tox target that will build
    PDF versions of our docs. As per the Train community goal:
    
      https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html
    
    Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
    to convert our SVGs.
    
    Story: 2006122
    Task: 35515
    Change-Id: I26cefda80d3234df68d7152b404e0a71da74ab90

commit be41721888913320bd448b8aaa4539f3ac6d4e7c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Sep 27 16:18:00 2019 -0700

Add experimental job to test upgrades from stein
    
    Also, correct the version that we check out when upgrading from stable
    branches.
    
    Change-Id: Ie733bc50466c66d6e6eb5c6bd42e42a05ef88798

commit 9a33365f064c2fbde732780982e3d324b488e677
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Sep 27 11:04:43 2019 -0700

py3: Allow percentages in configs
    
    Previously, configs like
    
        fallocate_reserve = 1%
    
    would cause a py3 backend server to fail to start, complaining like
    
        configparser.InterpolationSyntaxError: Error in file
        /etc/swift/object-server/1.conf.d: '%' must be followed
        by '%' or '(', found: '%'
    
    This could also come up in proxy-server configs, with things like
    percent signs in tempauth password.
    
    In general, we haven't really thought much about interpolation in
    configs. Python's default ConfigParser has always supported it, though,
    so we got it "for free". On py2, we didn't really have to think about
    it, since values like "1%" would pass through just fine. (It would blow
    up a SafeConfigParser, but a normal ConfigParser only does replacements
    when there's something like a "%(opt)s" in the value.)
    
    On py3, SafeConfigParser became ConfigParser, and the old interpolation
    mode (AFAICT) doesn't exist.
    
    Unfortunatley, since we "supported" interpolation, we have to assume
    there are deployments in the wild that use it, and try not to break
    them.  So, do what we can to mimic the py2 behavior.
    
    Change-Id: I0f9cecd11f00b522a8486972551cb30af151ce32
    Closes-Bug: #1844368

commit ad7f7da32d6f90aa49873f1021d18cd54daef102
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Aug 5 14:51:14 2019 -0700

py3: decode stdout from backgrounded servers
    
    Otherwise, when we go to print() it, we get a bunch of b"" strings.
    
    Change-Id: If62da0b4b34b9d1396b5838bf79ff494679f1ae3

commit e9cd9f74a5264f396783ca2a4548a3da7cee7bff
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Mon Aug 12 16:16:17 2019 +1000

sharder: Keep cleaving on empty shard ranges
    
    When a container is being cleaved there is a possiblity that we're
    dealing with an empty or near empty container created on a handoff node.
    These containers may have a valid list of shard ranges, so would need
    to cleave to the new shards.
    Currently, when using a `cleave_batch_size` that is smaller then the
    number of shard ranges on the cleaving container, these containers will
    have to take a few shard passes to shard, even though there maybe
    nothing in them.
    
    This is worse if a really large container is sharding, and due to being
    slow, error limitted a node causing a new container on a handoff
    location. This empty container would have a large number of shard ranges
    and could take a _very_ long time to shard away, slowing the process
    down.
    
    This patch eliminates the issue by detecting when no objects are
    returned for a shard range. The `_cleave_shard_range` method now
    returns 3 possible results:
    
      - CLEAVE_SUCCESS
      - CLEAVE_FAILED
      - CLEAVE_EMPTY
    
    They all are pretty self explanitory. When `CLEAVE_EMPTY` is returned
    the code will:
    
      - Log
      - Not replicate the empty temp shard container sitting in a
        handoff location
      - Not count the shard range in the `cleave_batch_size` count
      - Update the cleaving context so sharding can move forward
    
    If there already is a shard range DB existing on a handoff node to use
    then the sharder wont skip it, even if there are no objects, it'll
    replicate it and treat it as normal, including using a `cleave_batch_size`
    slot.
    
    Change-Id: Id338f6c3187f93454bcdf025a32a073284a4a159
    Closes-Bug: #1839355

commit f56071e57392573b7aea014bba6757a01a8a59ad
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Wed Sep 25 15:58:50 2019 -0500

Make sharding methods with only one job
    
    Change-Id: Id1e9a9ee316517923907bf0593e851448528c75c

commit 50255de0e3def868e958bfdf4aea9f4cc606e744
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Sep 23 16:21:36 2019 -0700

func tests: Add more UTF8 tests for versioning
    
    Change-Id: I7ac111bd8b57bd21c37f4c567a20e2c12957b2ff

commit 6271d88f9ed5e98f989a6739a75b268537fe0521
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Fri Aug 23 19:14:37 2019 +0200

Add func test for changing versionining modes
    
    Users are able to change versioning in a container
    from X-Versions-Location to X-History-Location, which affects
    how DELETEs are handled. We have some unit tests that check this
    behavior, but no functional tests.
    
    This patch adds a functional test that helps us understand and
    document how changing modes affects the handling of DELETE
    requests.
    
    Change-Id: I5dbe5bdca17e624963cb3a3daba3b240cbb4bec4

commit 9495bc0003817805750dd78f3d93dd1a237f1553
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Sep 19 16:52:41 2019 -0700

sharding: Update probe test to verify CleavingContext cleanup
    
    Change-Id: I219bbbfd6a3c7adcaf73f3ee14d71aadd183633b
    Related-Change: I1e502c328be16fca5f1cca2186b27a0545fecc16

commit 370ac4cd70489a49b2b6408638c9b35006f57053
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Sat Sep 21 16:06:24 2019 +1000

Sharding: Use the metadata timestamp as last_modified
    
    This is a follow up patch from the cleaning up cleave context's patch
    (patch 681970). Instead of tracking a last_modified timestamp, and storing
    it in the context metadata, use the timestamp we use when storing any
    metadata.
    
    Reducing duplication is nice, but there's a more significant reason to
    do this: affected container DBs can start getting cleaned up as soon as
    they're running the new code rather than needing to wait for an
    additional reclaim_age.
    
    Change-Id: I2cdbe11f06ffb5574e573c4a60ba4e5d41a00c50

commit 291873e784aeac30c2adcaaaca6ab43c2393b289
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Aug 15 14:33:06 2019 -0700

proxy: Don't trust Content-Length for chunked transfers
    
    Previously we'd
    - complain that a client disconnected even though they finished their
      chunked transfer just fine, and
    - on EC, send a X-Backend-Obj-Content-Length for pre-allocation even
      though Content-Length doesn't determine request body size.
    
    Change-Id: Ia80e595f713695cbb41dab575963f2cb9bebfa09
    Related-Bug: 1840507

commit 81a41da5420313f9cdb9c759bbb0f46c0d20c5af
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Fri Sep 13 16:16:06 2019 +1000

Sharding: Clean up old CleaveConext's during audit
    
    There is a sharding edge case where more CleaveContext are generated and
    stored in the sharding container DB. If this number get's high enough,
    like in the linked bug. If enough CleaveContects build up in the DB then
    this can lead to the 503's when attempting to list the container due to
    all the `X-Container-Sysmeta-Shard-Context-*` headers.
    
    This patch resolves this by tracking the a CleaveContext's last
    modified. And during the sharding audit, any context's that hasn't been
    touched after reclaim_age are deleted.
    
    This plus the skip empty ranges patches should improve these handoff
    shards.
    
    Change-Id: I1e502c328be16fca5f1cca2186b27a0545fecc16
    Closes-Bug: #1843313

commit 20fc16e8daa184ebadab9f49e0f76e7687a8cebd
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Tue Sep 17 18:57:35 2019 +0200

Close leaking opened requests
    
    Change-Id: I3d96022c01834c85e9795ea41d18b17624a33a19
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>

commit 9698b1bb957c1f646ac30fb64ec3528627fcee1c
Author: Thiago da Silva <thiagodasilva@gmail.com>
Date:   Tue Sep 17 16:52:55 2019 +0200

Skip test when object versioning is not enabled
    
    Change-Id: I671a6e4a3d1011dbbc2267b44134cfaf3380fb22

commit 75c9c636f2c637b0f36c705957f2204de6e405d0
Author: Ghanshyam Mann <gmann@ghanshyammann.com>
Date:   Tue Sep 17 04:47:45 2019 +0000

[train][goal] Run 'tempest-ipv6-only' job in gate
    
    As part of Train community goal 'Support IPv6-Only Deployments and Testing'[1],
    Tempest has defined the new job 'tempest-ipv6-only'(adding
    in Depends-On patch) which will deploy services on IPv6 and run smoke
    tests and IPv6 related tests present in Tempest.
    
    This job will be part of Nova, Neutron, Cinder, Keystone, Glance, Swift
    gate.
    
    Verification structure will be:
    - 'devstack-IPv6' deploy the service on IPv6
    - 'devstack-tempest-ipv6' run will verify the IPv6-only setting and listen address
    - 'tempest-ipv6-only' will run the smoke + IPv6 related test case.
    
    This commit adds the new job 'tempest-ipv6-only' run on gate.
    
    Story: #2005477
    Task: #35932
    
    [1] https://governance.openstack.org/tc/goals/train/ipv6-support-and-testing.html
    
    Change-Id: I78be2ee5a7f1e5d3188ece98d7d8324f1c9bd0e3

commit b4288b4aa6e6be2222f5f0e9ca8360c07040d5c0
Author: Nguyen Quoc Viet <nguyenqviet98@gmail.com>
Date:   Thu Sep 12 11:31:42 2019 +0700

versioned_writes: checks for SLO object before copy
    
    Previously, versioned_writes middleware copy an already existing
    object using PUT. However, SLO requires the additional query
    to properly update the object size when listing.
    
    Propose fix: In _put_versioned_obj - which is called when on
    creating version obj and also on restoring obj,
    if 'X-Object-Sysmeta-Slo-Size' header is present it will
    add needed headers for container to update obj size
    
    Added a new functional test case with size assertion for slo
    
    Change-Id: I47e0663e67daea8f1cf4eaf3c47e7c8429fd81bc
    Closes-Bug: #1840322

commit db8b0b6bc46a67b03af415d4e5e1429cc7d73bba
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri May 10 13:15:42 2019 -0500

Make ceph tests more portable
    
    Change-Id: If93325f2651a02f98f9d480c10bf7b849cc9617e

commit 3960df983b68cd5baa84cac9a4d0b61f08737c09
Author: Andreas Jaeger <aj@suse.com>
Date:   Fri Sep 13 09:38:22 2019 +0200

Remove unneeded Zuul branch matcher
    
    We have implicit branch matchers, so there's no need to add a check
    for not-ocata etc, a job is only run for the branch it's on - like
    master now.
    
    Remove it to not confuse Zuul when multiple branches matches and the job
    definition is different.
    
    Change-Id: I6a346c9141aad1aa8a7393c899d5571057073e7a

commit 49f62f6ab7fd1b833e9b5bfbcaafa4b45b592d34
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Sep 12 10:59:08 2019 -0700

bufferedhttp: ensure query params are properly quoted
    
    Recent versions of py27 [1] have begun raising InvalidURL if you try to
    include non-ASCII characters in the request path. This was observed
    recently in the periodic checks of stable/ocata and stable/pike. In
    particular, we would spin up some in-process servers in
    test.unit.proxy.test_server.TestSocketObjectVersions and do a container
    listing with a prefix param that included raw (unquoted) UTF-8. This
    query string would pass unmolested through the proxy, tripping the
    InvalidURL error when bufferedhttp called putrequest.
    
    More recent versions of Swift would not exhibit this particular failure,
    as the listing_formats middleware would force a decoding/re-encoding of
    the query string for account and container requests. However, object
    requests with errant query strings would likely be able to trip the same
    error.
    
    Swift on py3 should not exhibit this behavior, as we so
    thoroughly re-write the request line to avoid hitting
    https://bugs.python.org/issue33973.
    
    Now, always parse and re-encode the query string in bufferedhttp. This
    prevents any errors on object requests and cleans up any callers that
    might use bufferedhttp directly.
    
    [1] Anything after https://github.com/python/cpython/commit/bb8071a;
        see https://bugs.python.org/issue30458
    
    Closes-Bug: 1843816
    Change-Id: I73f84b96f164e6fc5d3cb890355871c26ed271a6
    Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
    Related-Change: Ie648f5c04d4415f3b620fb196fa567ce7575d522

commit 1ded0d6c8793ca3eca573c098cef78b5ae41f080
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Oct 11 15:23:39 2018 -0700

Allow arbitrary UTF-8 strings as delimiters in listings
    
    AWS seems to support this, so let's allow s3api to do it, too.
    
    Previously, S3 clients trying to use multi-character delimiters would
    get 500s back, because s3api didn't know how to handle the 412s that the
    container server would send.
    
    As long as we're adding support for container listings, may as well do
    it for accounts, too.
    
    Change-Id: I62032ddd50a3493b8b99a40fb48d840ac763d0e7
    Co-Authored-By: Thiago da Silva <thiagodasilva@gmail.com>
    Closes-Bug: #1797305

commit 4cafc3d656098d13c46cd83d94b44c8801c5eb2b
Author: CY Chiang <cychiang@cht.com.tw>
Date:   Thu Sep 5 16:09:23 2019 +0800

doc: Fix the swift middleware doc needs more info  to set s3 api
    
    Modify the AWS S3 Api section in middleware document.
    Add how to create ec2 credential and minimun configuration to use
    s3 api.
    
    Change-Id: Id4d614d8297662f16403fdfe526e14714a21249d
    Closes-Bug: #1842884

commit 1d7e1558b3b422073918b89df21f703215bd1e33
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Jul 16 17:01:19 2019 -0700

py3: (mostly) port probe tests
    
    There's still one problem, though: since swiftclient on py3 doesn't
    support non-ASCII characters in metadata names, none of the tests in
    TestReconstructorRebuildUTF8 will pass.
    
    Change-Id: I4ec879ade534e09c3a625414d8aa1f16fd600fa4

commit c71bb2506310438b011818a44449daea500863fd
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Aug 30 21:40:03 2019 -0700

diskfile: Add some argument validation
    
    Either all or none of account, container, and object should be provided.
    If we get some but not all, that's indicating some kind of a coding bug;
    there's a chance it may be benign, but it seems safer to fail early and
    loudly.
    
    Change-Id: Ia9a0ac28bde4b5dcbf6e979c131e61297577c120
    Related-Change: Ic2e29474505426dea77e178bf94d891f150d851b

commit e6e31410e093b426bfa5b9a2094be56c8406b6a2
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Aug 30 11:54:47 2019 -0700

Find .d pid files with swift-orphans
    
    Change-Id: I7a2f19862817abf15e51463bd124293730451602

commit 3e4efb7aa4662a5f915caab5bef3de6dd17e3e19
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Aug 29 16:55:27 2019 -0700

py3: Update Getting Started docs
    
    Change-Id: I94050c40585b397a9f7bab1e48650b89f70ab24d

commit 4d83b9b95e32038390dbdc66d93c36c929dbce2a
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Aug 15 14:33:06 2019 -0700

tests/py3: Improve header casing
    
    Previously, our unit tests with socket servers would let eventlet
    capitalize headers on the way out, which
    
    - isn't something we want to have eventlet do, because it
    - breaks unicode-in-header-names on py3, so it
    - is already disabled in swift.common.wsgi.run_server() for real servers.
    
    Include a test to make sure we don't forget about it in the future.
    
    Change-Id: I0156d0059092ed414b296c65fb70fc18533b074a

commit a32fb30c16062ea64488e918077d635645e33e47
Author: Ondřej Nový <ondrej.novy@firma.seznam.cz>
Date:   Mon Aug 20 10:11:15 2018 +0200

Use SOURCE_DATE_EPOCH in docs to make build reproducible
    
    Set copyright year and html_last_updated_fmt to SOURCE_DATE_EPOCH if
    it's set. See https://reproducible-builds.org/specs/source-date-epoch/
    
    This patch make build reproducible, see https://reproducible-builds.org/
    
    Change-Id: I730a8265ca2c70c639ef77a613908e84eb738b70

commit 2545372055922abd681ef665f9040590d2f5806c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Aug 16 20:37:10 2019 -0700

py3: Switch swift-dsvm-functional-py3 to run tests under py3
    
    Now that all of the func tests are ported, we may as well run all-py3.
    
    Change-Id: Ib9f75ca9efb46dc4c7730ad2718228ec7777c924

commit 74db3670607d952e597011eb07676aedff521b41
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Aug 7 16:16:57 2019 -0700

py3: Finish porting func tests
    
    We were (indirectly) importing swiftclient (and therefore requests and
    urllib3) before doing our eventlet monkey-patching. This would lead
    boto3 (which digs an SSLContext out of urllib3) to trip RecursionErrors
    on py3 similar to
    
       >>> from ssl import SSLContext, PROTOCOL_SSLv23
       >>> import eventlet
       >>> eventlet.monkey_patch(socket=True)
       >>> SSLContext(PROTOCOL_SSLv23).options |= 0
       Traceback (most recent call last):
         File "<stdin>", line 1, in <module>
         File "/usr/lib/python3.6/ssl.py", line 465, in options
           super(SSLContext, SSLContext).options.__set__(self, value)
         File "/usr/lib/python3.6/ssl.py", line 465, in options
           super(SSLContext, SSLContext).options.__set__(self, value)
         File "/usr/lib/python3.6/ssl.py", line 465, in options
           super(SSLContext, SSLContext).options.__set__(self, value)
         [Previous line repeated 330 more times]
       RecursionError: maximum recursion depth exceeded while calling a Python object
    
    Change-Id: I4bb59edd87336597791416c4f2a096efe0e72fe3

commit 3750285bc863f8b6b56ba9526b028ee9cddcf04b
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Jul 16 16:24:14 2019 -0700

py3: fix up listings on sharded containers
    
    We were playing a little fast & loose with types before; as a result,
    marker/end_marker weren't quite working right. In particular, we were
    checking whether a WSGI string was contained in a shard range, while
    ShardRange assumes all comparisons are against native strings.
    
    Now, get everything to native strings before making comparisons, and
    get them back to wsgi when we shove them in the params dict.
    
    Change-Id: Iddf9e089ef95dc709ab76dc58952a776246991fd

commit a48dd1950d2999cb7fdc2856a827da4780715b1e
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Aug 5 14:48:54 2019 -0700

Allow non-default domain to be used in func tests
    
    Change-Id: I7afa7e367103bb9caaf74788a49cd055eca53cf6

commit f1b44b199a064c3715c4b0e1e4067ec8235cf18d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Mar 29 13:54:30 2019 -0700

s3api: paginate listings when aborting MPUs
    
    Even when your cluster's configured funny, like your
    container_listing_limit is too low, or your max_manifest_segments and
    max_upload_part_num are too high, an abort should (attempt to) clean up
    *all* segments.
    
    Change-Id: I5a57f919cc74ddb08bbb35a7d852fbc1457185e8

commit c0035ed82e52756c9c04097fabba561a86da200a
Author: CY Chiang <cychiang@cht.com.tw>
Date:   Tue Jul 30 11:42:45 2019 +0800

Update the bandit.yaml available tests list
    
    According to the bandit current version document,
    the B109 and B111 plugin has been removed.
    And Add the following tests:
    Complete Test Plugin Listing: B507, B610, B611, B703
    Blacklist Plugins Listing: B322, B323, B325, B413, B414
    Reference URL: https://bandit.readthedocs.io/en/latest/plugins/index.html
    
    Change-Id: I5e9365f9147776d7d90c6ba889acbde3c0e6c19d
    Closes-Bug: #1838361

commit 6853616aeaa7a6b14fd1ae99a507ab1761d16609
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jul 12 15:17:34 2019 -0700

ring: Track more properties of the ring
    
    Plumb the version from the ringbuilder through to the metadata at the
    start of the ring. Recover this (if available) when running
    
        swift-ring-builder <ring> write_builder
    
    When we load the ring, track the count and MD5 of the bytes off disk, as
    well as the number of uncompressed bytes.
    
    Expose all this new information as properties on the Ring, along with
    
      - device_count (number of non-None entries in self._devs),
      - weighted_device_count (number of devices that have weight), and
      - assigned_device_count (number of devices that actually have
        partition assignments).
    
    Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
    Change-Id: I73deaf6f1d9c1d37630c37c02c597b8812592351

commit 0fec28ab155276d099d1d4c9fd377f3da539077b
Author: zhufl <zhu.fanglei@zte.com.cn>
Date:   Wed Jul 3 16:41:38 2019 +0800

Fix invalid assert states
    
    This is to fix invalid assert states like:
        self.assertTrue('sync_point2: 5', lines.pop().strip())
        self.assertTrue('sync_point1: 5', lines.pop().strip())
        self.assertTrue('bytes: 1100', lines.pop().strip())
        self.assertTrue('deletes: 2', lines.pop().strip())
        self.assertTrue('puts: 3', lines.pop().strip())
        self.assertTrue('1', jobs_to_delete[0]['partition'])
    in which assertEqual should be used.
    
    Change-Id: Ide5af2ae68fae0e5d6eb5c233a24388bb9942144

commit 03512e001d95adadfea147e8a4051fce0aa9dfca
Author: pengyuesheng <pengyuesheng@gohighsec.com>
Date:   Wed Jul 3 15:06:31 2019 +0800

Update the constraints url
    
    For more detail, see http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html
    
    Change-Id: I95114c4aa670c07491d5a15db2341f65cb0d1344

commit 5270da86e6eead273f58a24cba65b951550e3037
Author: pengyuesheng <pengyuesheng@gohighsec.com>
Date:   Tue Jul 2 10:59:28 2019 +0800

Add python3 to setup.cfg
    
    Change-Id: I5dd57aad794c050c44e328c43346be0063170492

commit 4aa71aa25caed34f36fafe2de025425aa1d1e0b2
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Tue Oct 9 16:42:18 2018 -0700

We don't have to keep the retrieved token anymore
    
    Since the change in s3_token_middleware to retrieve the auth info
    from keystone directly, now, we don't need to have any tokens provided
    by keystone in the request header as X-Auth-Token.
    
    Note that this makes the pipeline ordering change documented in the
    related changes mandatory, even when working with a v2 Keystone server.
    
    Change-Id: I7c251a758dfc1fedb3fb61e351de305b431afa79
    Related-Change: I21e38884a2aefbb94b76c76deccd815f01db7362
    Related-Change: Ic9af387b9192f285f0f486e7171eefb23968007e

commit eed76d8bed446518bff2ca4af18259f7637c430e
Author: arzhna <arzhna@gmail.com>
Date:   Wed Nov 28 11:15:05 2018 +0900

Fix a potential bug
    
    In the class method from_hash_dir(), the arguments to input when creating an instance of the BaseDiskFile class are incorrect.
    The __init__() method of BaseDiskFile class receive the arguments in order of mgr, device_path, partition and etc.
    However, in from_hash_dir() method, the order of arguments are mgr, device_path, None and partition
    The class method from_hash_dir() is used by the Object Auditor.
    If the partition argument is used in the new DiskFile implementations, exception may occur.
    It will be cause object auditing to failed and the object will be quarantine by the Object Auditor.
    
    Closes-Bug: #1805539
    Change-Id: Ic2e29474505426dea77e178bf94d891f150d851b

tags:

added: in-feature-losf

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-18: Fix proposed to swift (stable/stein)

#7

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/689567

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-23: Fix merged to swift (stable/stein)

#8

Reviewed: https://review.opendev.org/689567
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=6e9e922cd6d6f7144605c5e959d3ff64f4bbc461
Submitter: Zuul
Branch: stable/stein

commit 6e9e922cd6d6f7144605c5e959d3ff64f4bbc461
Author: Matthew Oliver <email address hidden>
Date: Fri Sep 13 16:16:06 2019 +1000

Sharding: Clean up old CleaveConext's during audit

This is a combination of 4 commits:

==========

Sharding: Clean up old CleaveConext's during audit

    There is a sharding edge case where more CleaveContext are generated and
    stored in the sharding container DB. If this number get's high enough,
    like in the linked bug. If enough CleaveContects build up in the DB then
    this can lead to the 503's when attempting to list the container due to
    all the `X-Container-Sysmeta-Shard-Context-*` headers.

    This patch resolves this by tracking the a CleaveContext's last
    modified. And during the sharding audit, any context's that hasn't been
    touched after reclaim_age are deleted.

This plus the skip empty ranges patches should improve these handoff
shards.

(cherry picked from commit 81a41da5420313f9cdb9c759bbb0f46c0d20c5af)

----------

Sharding: Use the metadata timestamp as last_modified

    This is a follow up patch from the cleaning up cleave context's patch
    (patch 681970). Instead of tracking a last_modified timestamp, and storing
    it in the context metadata, use the timestamp we use when storing any
    metadata.

    Reducing duplication is nice, but there's a more significant reason to
    do this: affected container DBs can start getting cleaned up as soon as
    they're running the new code rather than needing to wait for an
    additional reclaim_age.

(cherry picked from commit 370ac4cd70489a49b2b6408638c9b35006f57053)

----------

Make sharding methods with only one job

(cherry picked from commit f56071e57392573b7aea014bba6757a01a8a59ad)

----------

sharding: Update probe test to verify CleavingContext cleanup

(cherry picked from commit 9495bc0003817805750dd78f3d93dd1a237f1553)

==========

    Co-Authored-By: Clay Gerrard <email address hidden>
    Co-Authored-By: Tim Burke <email address hidden>
    Change-Id: I1e502c328be16fca5f1cca2186b27a0545fecc16
    Closes-Bug: #1843313

Reviewed:  https://review.opendev.org/689567
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=6e9e922cd6d6f7144605c5e959d3ff64f4bbc461
Submitter: Zuul
Branch:    stable/stein

commit 6e9e922cd6d6f7144605c5e959d3ff64f4bbc461
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Fri Sep 13 16:16:06 2019 +1000

Sharding: Clean up old CleaveConext's during audit
    
    This is a combination of 4 commits:
    
    ==========
    
    Sharding: Clean up old CleaveConext's during audit
    
    There is a sharding edge case where more CleaveContext are generated and
    stored in the sharding container DB. If this number get's high enough,
    like in the linked bug. If enough CleaveContects build up in the DB then
    this can lead to the 503's when attempting to list the container due to
    all the `X-Container-Sysmeta-Shard-Context-*` headers.
    
    This patch resolves this by tracking the a CleaveContext's last
    modified. And during the sharding audit, any context's that hasn't been
    touched after reclaim_age are deleted.
    
    This plus the skip empty ranges patches should improve these handoff
    shards.
    
    (cherry picked from commit 81a41da5420313f9cdb9c759bbb0f46c0d20c5af)
    
    ----------
    
    Sharding: Use the metadata timestamp as last_modified
    
    This is a follow up patch from the cleaning up cleave context's patch
    (patch 681970). Instead of tracking a last_modified timestamp, and storing
    it in the context metadata, use the timestamp we use when storing any
    metadata.
    
    Reducing duplication is nice, but there's a more significant reason to
    do this: affected container DBs can start getting cleaned up as soon as
    they're running the new code rather than needing to wait for an
    additional reclaim_age.
    
    (cherry picked from commit 370ac4cd70489a49b2b6408638c9b35006f57053)
    
    ----------
    
    Make sharding methods with only one job
    
    (cherry picked from commit f56071e57392573b7aea014bba6757a01a8a59ad)
    
    ----------
    
    sharding: Update probe test to verify CleavingContext cleanup
    
    (cherry picked from commit 9495bc0003817805750dd78f3d93dd1a237f1553)
    
    ==========
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    Change-Id: I1e502c328be16fca5f1cca2186b27a0545fecc16
    Closes-Bug: #1843313

tags:

added: in-stable-stein

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-11-26: Fix included in openstack/swift 2.21.1

#9

This issue was fixed in the openstack/swift 2.21.1 release.

OpenStack Object Storage (swift)

Sharding handoffs creates a ton of container-server headers

Bug Description

Other bug subscribers

Patches

Remote bug watches

OpenStack Object Storage (swift)

Sharding handoffs creates a *ton* of container-server headers

Bug Description

Other bug subscribers

Patches

Remote bug watches

Sharding handoffs creates a ton of container-server headers