Reviewed:  https://review.openstack.org/529226
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=cbdf0514f001abf2d6a6fd8669fcc2d413c8136d
Submitter: Zuul
Branch:    feature/s3api

commit 1984353f0d6db7512e4ea147ecad9e14dfb318d4
Author: Alistair Coles <email address hidden>
Date:   Fri Dec 15 12:36:47 2017 +0000

    Move symlink versioning functional test
    
    The functional test for versioning symlinks is better located in
    test_versioned_writes where it can be added to
    TestObjectVersioning. This saves duplicated versioned_writes specific
    setup code in test_symlink, and has the benefit of the test being
    repeated for each of the versioned writes test subclasses.  With a
    small refactor this includes the test now running with
    x-history-location mode as well as x-versions-location mode.
    
    Related-Change: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18
    Change-Id: If215446c558b61c1a8aea37ce6be8fcb5a9ea2f4

commit c579e99126b61466fb3b1628170cbca37ccacce3
Author: Kota Tsuyuzaki <email address hidden>
Date:   Wed Dec 13 06:04:40 2017 +0000

    Add more assertions for Symlink + Copy unit tests
    
    Related-Change-Id: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18
    Change-Id: Ib4c8f0c83537b74bbdec8c2dd6acc99c039faa67

commit 097e975938befb939fa6821f50c061e2c7f42cef
Author: Clay Gerrard <email address hidden>
Date:   Thu Dec 14 16:17:29 2017 -0800

    Remove symlink from xml listing response
    
    We've had some problems with brittle XML clients in the past - it might
    be safer to ask that clients that need symlink keys in listings from
    containers request in JSON.
    
    Change-Id: I4ac7457f3ccb10f9e471ec6dc6f0869d71712878

commit 7647defb0f201cbf85baef6404067bb0eb27321f
Author: Clay Gerrard <email address hidden>
Date:   Thu Dec 14 12:56:49 2017 -0800

    rename utils function less like stdlib
    
    Related-Change-Id: I3436bf3724884fe252c6cb603243c1195f67b701
    Change-Id: I74199c62b46e4db93a76760ebf91d84e3e1e3cfc

commit b342a8147c38ebcf02c3ba21fd09ded4ca49f69b
Author: Alistair Coles <email address hidden>
Date:   Tue Dec 12 17:32:55 2017 +0000

    Assert X-Newest and X-Backend headers are propagated to symlink target
    
    Adds some assertions to verify that X-Newest and X-Backend-* headers are
    copied from an original object GET requets to the symlink target
    request.
    
    Change-Id: Idce92edd002dec34f5dbc5d3c28a4cbbd2fbdc60
    Related-Change: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18

commit 1d5cf3e73067751c8c8fd4f7f58c205db9b877a1
Author: Clay Gerrard <email address hidden>
Date:   Thu Dec 14 12:15:19 2017 -0800

    add symlink to probetest for reconciler
    
    Change-Id: Ib2c5616f2965ab92b1c76d573e869206c91464c6

commit 90134ee968f6f6442eedb5548ee292fc03c77c2a
Author: Clay Gerrard <email address hidden>
Date:   Thu Dec 14 12:09:04 2017 -0800

    add symlink to container sync default and sample config
    
    Change-Id: I83e51daa994b9527eccbb8a88952c630aacd39c3

commit 8df263184be136f9ab203a2971b4f47a52f8b431
Author: Alistair Coles <email address hidden>
Date:   Tue Dec 12 17:26:03 2017 +0000

    Symlink doc clean up
    
    Cleanup for docs and docstrings.
    
    Related-Change: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18
    Change-Id: Ie8de0565dfaca5bd8a5693a75e6ee14ded5b7161

commit 99b89aea1051208e3d71afa35fcd62035e702628
Author: Robert Francis <email address hidden>
Date:   Wed Oct 7 15:14:58 2015 -0400

    Symlink implementation.
    
    Add a symbolic link ("symlink") object support to Swift. This
    object will reference another object. GET and HEAD
    requests for a symlink object will operate on the referenced object.
    DELETE and PUT requests for a symlink object will operate on the
    symlink object, not the referenced object, and will delete or
    overwrite it, respectively.
    POST requests are *not* forwarded to the referenced object and should
    be sent directly. POST requests sent to a symlink object will
    result in a 307 Error.
    
    Historical information on symlink design can be found here:
    https://github.com/openstack/swift-specs/blob/master/specs/in_progress/symlinks.rst.
    https://etherpad.openstack.org/p/swift_symlinks
    
    Co-Authored-By: Thiago da Silva <email address hidden>
    Co-Authored-By: Janie Richling <email address hidden>
    Co-Authored-By: Kazuhiro MIYAHARA <email address hidden>
    Co-Authored-By: Kota Tsuyuzaki <email address hidden>
    
    Change-Id: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18
    Signed-off-by: Thiago da Silva <email address hidden>

commit b86bf15a644db4438770801a312fe074a09c91ef
Author: Thiago da Silva <email address hidden>
Date:   Mon Dec 11 06:59:59 2017 -0500

    remove left-over old doc in config file
    
    Removing the rest of fast post documentation
    from proxy configuration file.
    
    Change-Id: I6108be20606f256a020a8583878e2302e3c98ff1

commit 8182fa75a167d034e29c04680502ea9877592c9a
Author: Samuel Merritt <email address hidden>
Date:   Fri Dec 8 14:08:25 2017 -0800

    Fix small error in a doc string
    
    Change-Id: I1c743fdea637ce047d09a49db0a43b2eb37305fa

commit dd113ab25a3089fa19c8e824c2d89db0ca3db0fa
Author: Alistair Coles <email address hidden>
Date:   Tue Dec 5 10:36:38 2017 -0800

    Refactor proxy-server conf loading to a helper function
    
    There were two middlewares using a common pattern to load
    the proxy-server app config section. The existing pattern
    fails to recognise option overrides that are declared using
    paste-deploy's 'set' notation, as illustrated by the change
    to test_dlo.py in this patch.
    
    This patch replaces the existing code with a helper function
    that loads the proxy-server config using the paste-deploy loader.
    The resulting config dict is therefore exactly the same as that
    used to initialise the proxy-server app.
    
    Change-Id: Ib58ce03e2010f41e7eb11f1a6dc78b0b7f55d466

commit 84ea58b8c81814a3c4d450145bfb9e70166dd60b
Author: Christopher Bartz <email address hidden>
Date:   Mon Dec 4 14:53:44 2017 +0100

    Ringbuilder: Forbid writing empty rings
    
    Swift definitely can't make any use of empty rings, so it should
    not be allowed to write them.
    
    Replace warning with an error message & error exit.
    
    Change-Id: I3a1b86368d363e67d1f91d7d8af4b391a0a53fff
    Closes-Bug: #1396841

commit fc12d63c76ffd51a5d524caa92b78d29bc4e6a7d
Author: Alistair Coles <email address hidden>
Date:   Wed Dec 6 10:29:52 2017 -0800

    Remove repeated text from deployment guide
    
    Fix what appears to be a cut and paste error.
    
    Change-Id: Iccf97ebbf75c8f97095a4493ea6a8beb074df099

commit bc2e03d1a6cb1411a2eee8218cce53f951100eb1
Author: Alistair Coles <email address hidden>
Date:   Tue Dec 5 13:45:18 2017 -0800

    Add --swift-versions option to swift-recon man page
    
    Related-Change: I3c2e569f0c44168333251bb58bab4b5582e15a45
    
    Change-Id: I9776c0919164e48ac445eacf7d897a23ef8e4572

commit 206f674014fb94a63ebdae91e3eca38956055b07
Author: Peter Lisák <email address hidden>
Date:   Tue Dec 20 16:27:36 2016 +0100

    Added swift version to recon cli
    
    Show swift version and check if the same on hosts by using
    `swift-recon --swift-version`.
    
    Ex:
    $ swift-recon --swift-versions
    
    Versions matched (2.7.1.dev144), 0 error[s] while checking hosts.
    
    or if differs
    
    Versions not matched (2.7.1.dev144, 2.7.1.dev145),
    0 error[s] while checking hosts.
    
    Change-Id: I3c2e569f0c44168333251bb58bab4b5582e15a45

commit b808c806b4e7000aed45a0026dd7407cd1616891
Author: Tim Burke <email address hidden>
Date:   Wed Nov 29 17:24:40 2017 -0800

    Remove testTooLongName tests
    
    These are better-covered by TestContainer.testContainerNameLimit and
    TestFile.testNameLimit in the same file.
    
    Change-Id: Ice48bc6492648613bc743b474d40892d7e4dcc64

commit 924f0d28e9670163dc47206cc1472ef228fd0082
Author: Tim Burke <email address hidden>
Date:   Wed Nov 22 16:51:06 2017 -0800

    dlo: Move conn2 business to the one test that uses it
    
    ...and skip it if we don't have the required user
    
    Change-Id: I8700a587d5b8acff1f0255529b6ddaeadaaa6548

commit 3135878d2fe9909f49fcadeeb9cc6c6933d06127
Author: Robin Naundorf <email address hidden>
Date:   Mon Nov 20 12:21:30 2017 +0100

    Fix small formatting issues in middleware docs
    
    TrivialFix
    
    Change-Id: I0ebc12e257b630696a82890178436dcd1eef0c36

commit 4f3abec614e90dd1217c70284b7469024c362e62
Author: Robin Naundorf <email address hidden>
Date:   Mon Nov 20 10:59:05 2017 +0100

    Fix small formatting issue
    
    TrivialFix
    
    Change-Id: Iacd341a1f3dbf43c6fc2ebcf546aed54f866face

commit de818611085d5781fa449928e90e891d863c6768
Author: Robin Naundorf <email address hidden>
Date:   Fri Nov 17 13:30:08 2017 +0100

    Fix formatting of SAIO docs
    
    * Fix formatting error in SAIO docs so that list is displaying
      correct
    
    Change-Id: If88869290252a2c86f60fdf89d78ef4a418ae23d

commit c3e73b64f8c2d3b700b8bd39339c60a096d50cb8
Author: Andreas Jaeger <email address hidden>
Date:   Fri Nov 17 10:28:20 2017 +0100

    Remove setting of version/release from releasenotes
    
    Release notes are version independent, so remove version/release
    values. We've found that projects now require the service package
    to be installed in order to build release notes, and this is entirely
    due to the current convention of pulling in the version information.
    
    Release notes should not need installation in order to build, so this
    unnecessary version setting needs to be removed.
    
    This is needed for new release notes publishing, see
    I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting
    at
    http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html
    .
    
    Change-Id: I1442656b3e4ab84ee5c219f5f4959bd7f41f01fe

commit 1962b18c0c4e0069ccd3063e06a5d491672db975
Author: Kota Tsuyuzaki <email address hidden>
Date:   Tue Nov 14 11:42:11 2017 +0900

    Remove unnecessary commented lines
    
    Change-Id: Ifb7e697851e4e6979fe22176c7a7ca74699c75a6

commit 396380f3406bdcd842e85a66cb346bf219cbb4f3
Author: Tim Burke <email address hidden>
Date:   Thu Nov 2 11:43:16 2017 -0700

    Better handle missing files in _construct_from_data_file
    
    There's a window between when we list the files on disk and when we actually
    try to open the .data file where another process could delete it. That should
    raise a DiskFileNotExist error, not an IOError.
    
    Change-Id: I1b43fef35949cb6f71997874e4e6b7646eeec8fe
    Closes-Bug: 1712662

commit 62d21eceb7ebd061d527339b85d5d9e24fa13a2b
Author: Samuel Merritt <email address hidden>
Date:   Thu Oct 19 17:13:00 2017 -0700

    Add metadata checksums to old objects in auditor.
    
    When the object auditor examines an object, it will now add any
    missing metadata checksums. This goes for both .data and .meta files,
    but not .ts files, as tombstones don't live very long anyway.
    
    Change-Id: I9417a8b0cc5099470845c0504c834746188d89e8

commit 6990d0209db3f7cb836d5df04fd39953430afd3a
Author: John Dickinson <email address hidden>
Date:   Fri Nov 10 11:03:18 2017 -0800

    added some people to core emeritus in the authors file
    
    Change-Id: I689490fd4e8d1256fc3cedf113cbd815062ae541

commit 1ff3f1e3e877091f372a9109187a3ce4cf4414dd
Author: John Dickinson <email address hidden>
Date:   Thu Nov 2 12:46:14 2017 -0700

    authors changelog for 2.16.0
    
    Change-Id: I2c1f4a55d2813a63275072a3aaaea69f6808125a

commit 5a06e3da3bb667f82872852f948fa451070f08da
Author: Steve Kowalik <email address hidden>
Date:   Tue Nov 7 12:05:27 2017 +1100

    No longer import nose
    
    Since Python 2.7, unittest in the standard library has included mulitple
    facilities for skipping tests by decorators as well as an exception.
    Switch to that directly, rather than importing nose.
    
    Change-Id: I4009033473ea24f0d0faed3670db844f40051f30

commit 53d92c18253355f15e3ca0fd07e13608b192dce8
Author: Alistair Coles <email address hidden>
Date:   Mon Nov 6 09:46:57 2017 +0000

    Increase connection timeout for object controller tests
    
    Intermittent test failures due to timeout of *mocked*
    connections have been reported, so increase the
    connection timeout in the test app.
    
    Closes-Bug: #1730373
    Change-Id: I3c91bdff65f8aa06a72f6ad5ec00ee67890b8e61

commit 728b4ba14008f59ffe64fd28bc061c12a0c64467
Author: Samuel Merritt <email address hidden>
Date:   Thu Jun 30 16:52:58 2016 -0700

    Add checksum to object extended attributes
    
    Currently, our integrity checking for objects is pretty weak when it
    comes to object metadata. If the extended attributes on a .data or
    .meta file get corrupted in such a way that we can still unpickle it,
    we don't have anything that detects that.
    
    This could be especially bad with encrypted etags; if the encrypted
    etag (X-Object-Sysmeta-Crypto-Etag or whatever it is) gets some bits
    flipped, then we'll cheerfully decrypt the cipherjunk into plainjunk,
    then send it to the client. Net effect is that the client sees a GET
    response with an ETag that doesn't match the MD5 of the object *and*
    Swift has no way of detecting and quarantining this object.
    
    Note that, with an unencrypted object, if the ETag metadatum gets
    mangled, then the object will be quarantined by the object server or
    auditor, whichever notices first.
    
    As part of this commit, I also ripped out some mocking of
    getxattr/setxattr in tests. It appears to be there to allow unit tests
    to run on systems where /tmp doesn't support xattrs. However, since
    the mock is keyed off of inode number and inode numbers get re-used,
    there's lots of leakage between different test runs. On a real FS,
    unlinking a file and then creating a new one of the same name will
    also reset the xattrs; this isn't the case with the mock.
    
    The mock was pretty old; Ubuntu 12.04 and up all support xattrs in
    /tmp, and recent Red Hat / CentOS releases do too. The xattr mock was
    added in 2011; maybe it was to support Ubuntu Lucid Lynx?
    
    Bonus: now you can pause a test with the debugger, inspect its files
    in /tmp, and actually see the xattrs along with the data.
    
    Since this patch now uses a real filesystem for testing filesystem
    operations, tests are skipped if the underlying filesystem does not
    support setting xattrs (eg tmpfs or more than 4k of xattrs on ext4).
    
    References to "/tmp" have been replaced with calls to
    tempfile.gettempdir(). This will allow setting the TMPDIR envvar in
    test setup and getting an XFS filesystem instead of ext4 or tmpfs.
    
    THIS PATCH SIGNIFICANTLY CHANGES TESTING ENVIRONMENTS
    
    With this patch, every test environment will require TMPDIR to be
    using a filesystem that supports at least 4k of extended attributes.
    Neither ext4 nor tempfs support this. XFS is recommended.
    
    So why all the SkipTests? Why not simply raise an error? We still need
    the tests to run on the base image for OpenStack's CI system. Since
    we were previously mocking out xattr, there wasn't a problem, but we
    also weren't actually testing anything. This patch adds functionality
    to validate xattr data, so we need to drop the mock.
    
    `test.unit.skip_if_no_xattrs()` is also imported into `test.functional`
    so that functional tests can import it from the functional test
    namespace.
    
    The related OpenStack CI infrastructure changes are made in
    https://review.openstack.org/#/c/394600/.
    
    Co-Authored-By: John Dickinson <email address hidden>
    
    Change-Id: I98a37c0d451f4960b7a12f648e4405c6c6716808

commit 31dd95ee91589be937565692a1610ba49db29706
Author: Tim Burke <email address hidden>
Date:   Thu Oct 12 22:27:35 2017 +0000

    Add base64decode function to common/utils
    
    Keymaster middleware does some nice input validation on
    base64-encoded strings; pull that out somewhere common so
    other things (like SLOs with inlined data) can use it, too.
    
    Change-Id: I3436bf3724884fe252c6cb603243c1195f67b701

commit 93e0287a827c0025c1522517b0e288ec04f12326
Author: Alistair Coles <email address hidden>
Date:   Tue Oct 24 16:06:26 2017 +0100

    Require lock_path limit to be a positive int
    
    Currently if limit=0 is passed to lock_path then the method will time
    out and never acquire a lock which is reasonable but not useful. This
    is a potential pitfall given that in other contexts, for example
    diskfile's replication_lock, a concurrency value of 0 has the meaning
    'no limit'. It would be easy to erroneously assume that the same
    semantic holds for lock_path.
    
    To avoid that pitfall, this patch makes it an error to pass limit<1 to
    lock_path.
    
    Related-Change: I3c3193344c7a57a8a4fc7932d1b10e702efd3572
    Change-Id: I9ea7ee5b93e3d6924bff9790141b107b53f77883

commit e0244dd30d1974db1c522c3b4a2982eaed6c5297
Author: Alistair Coles <email address hidden>
Date:   Tue Oct 24 13:03:05 2017 +0100

    Cleanup lock_path and replication_concurrency_per_device tests
    
    use assertRaises where applicable; use the with_tempdir
    decorator
    
    Related-Change: I3c3193344c7a57a8a4fc7932d1b10e702efd3572
    Change-Id: Ie83584fc22a5ec6e0a568e39c90caf577da29497