Reviewed:  https://review.openstack.org/154627
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=e852ca94cc7efe6da77d8d72f4155ab151356fd4
Submitter: Jenkins
Branch:    feature/ec

commit f96b8e412db73721660aa8247430db76c3149508
Author: Janie Richling <email address hidden>
Date:   Mon Feb 9 18:16:25 2015 -0600

    Included sysmeta in the object info
    
    The cached info object dict did not include
    the sysmeta.  This patch fixes that, and adds
    a unit test.
    
    Change-Id: I092200e76586af322ed4ff7d194a1034b1ca0433

commit b54532ca0581738c7c4622cd32e936088e9adf11
Author: OpenStack Proposal Bot <email address hidden>
Date:   Fri Feb 6 06:10:34 2015 +0000

    Imported Translations from Transifex
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I4b8411f84784a2825a24946c51f4c08ad01febc2

commit b45b83fb00917b729253804443094d97d39ef78f
Author: John Dickinson <email address hidden>
Date:   Thu Feb 5 11:01:02 2015 -0800

    Correct the config default for delay_auth_decision
    
    Updated proxy-server.conf-sample with the correct default. Also
    updated the note on the overview-auth doc page.
    
    Change-Id: I5cd62a7a118a28f7b58f47b8d8d4d963f6bc7347

commit 74563ece4782d16bf5a66db63d7c0c718a69db38
Author: Mahati Chamarthy <email address hidden>
Date:   Sat Jan 31 01:40:22 2015 +0530

    Tests for the base class of storage nodes
    
    Change-Id: I8866e2360e30a239d5f6b4a5ed92344291184c2a

commit b5f8c594f98ec943b73633f7418695880b315b6c
Author: Nicolas Trangez <email address hidden>
Date:   Wed Feb 4 16:54:46 2015 +0100

    Add `swift-scality-backend` to associated projects
    
    Change-Id: I7fd56c7cf5b7634224b8a2876258cf1f6be447f1

commit 2eba998a7ccde6a490d927f2489838e2954f712d
Author: John Dickinson <email address hidden>
Date:   Tue Feb 3 22:12:03 2015 -0800

    added swift-ui browser to associated projects
    
    Change-Id: I23abd014b21d1a968fe7352f3915f02e3d4d47cf

commit d85371c31995c2d672f363e511708ba6e736fc34
Author: Alistair Coles <email address hidden>
Date:   Tue Feb 3 18:24:41 2015 +0000

    Don't skip account acl functional tests
    
    The functional tests covering account acls are skipped
    if keystoneauth is in the pipeline, even if keystone auth
    is not being used. Devstack configures the pipeline to have
    both tempauth and keystoneauth, so these tests are always
    skipped.
    
    This patch changes the condition for skipping account acl
    to be based on tempauth being the auth service configured
    for the tests.
    
    Change-Id: I378ec6aa0ba52d37a33796057e59a9ebfcab2574

commit 5a0f8f14029121b35f03c2518347fafea8a4f28f
Author: Alistair Coles <email address hidden>
Date:   Tue Feb 3 11:57:06 2015 +0000

    Update auth_token section in documentation
    
    Bring docs in line with changes to auth_token config
    defaults made in I7076fa03ab531cbb1114918f75113620b65590dc
    
    Change-Id: Ia21685ebd1f3ed7bdba9de2ebac9fdcce8495949

commit a270dca23989b5cf7b59cf1d0e285bc736717a44
Author: Takashi Kajinami <email address hidden>
Date:   Fri Jan 23 20:16:20 2015 +0900

    Prevent redundant commenting by drive-audit
    
    The drive-audit detects error log about a device and comments out it
    in /etc/fstab. When the error log is generated several times, it
    comments out the line for each time.
    This patch makes drive-audit to check if the device is already
    commented out, and prevents redundant commenting out.
    
    Change-Id: Ia542d35b58552dde0f324bb9c42531f98c9058fa

commit efb39a5665f9c9f2649aac257d0f617479dea4ce
Author: Hisashi Osanai <email address hidden>
Date:   Tue Dec 9 04:44:52 2014 +0900

    Allow hostnames for nodes in Rings
    
    This change modifies the swift-ring-builder and introduces new format
    of sub-commands (search, list_parts, set_weight, set_info and remove)
    in addition to add sub-command so that hostnames can be used in place
    of an ip-address for the sub-commands.
    The account reaper, container synchronizer, and replicators were also
    updated so that they still have a way to identify a particular device
    as being "local".
    
    Previously this was Change-Id:
    Ie471902413002872fc6755bacd36af3b9c613b74
    
    Change-Id: Ieff583ffb932133e3820744a3f8f9f491686b08d
    Co-Authored-By: Alex Pecoraro <email address hidden>
    Implements: blueprint allow-hostnames-for-nodes-in-rings

commit d8fdbc2b2d755c0774b3a15538d40c83daf61b18
Author: sarvesh-ranjan <email address hidden>
Date:   Thu Jan 29 18:20:19 2015 -0800

    Typos fixed
    
    Change-Id: I2c216a870ce299039dec9948dcdef3de0721b4da

commit c8f02a88740183d449376fc2d8a1befb7aa7a5a7
Author: Mahati Chamarthy <email address hidden>
Date:   Wed Jan 28 22:29:22 2015 +0530

    Add server type in OPTIONS response
    
    Change-Id: I731872eaf2c878476525aa05fd60b9121be43e29

commit cec00660cbcb11086acd1906bcb0a206a846509c
Author: Bob Ball <email address hidden>
Date:   Wed Jan 28 16:07:58 2015 +0000

    Remove deprecated config variables
    
    I1f8f5064ea8028af60f167df9b97e215cdadba44 deprecated auth_host etc but the default
    config still used them.  Ieac26806bd420aa08fc79bbc6a11eb6a1c15c7df then switched
    devstack to using the new variables, but if the old variables still existed in the
    default config, some installations were broken (e.g. XenServer CI)
    
    Partial-bug: 1415795
    
    Change-Id: I7076fa03ab531cbb1114918f75113620b65590dc

commit 562f7e8906140ece51187854ed535a3ef0798a00
Author: Pete Zaitcev <email address hidden>
Date:   Wed Jan 28 12:21:25 2015 -0700

    Allow set_overload to take value as percent
    
    ...and output overload as a percent like dispersion and balance.
    
    Also raise a warning if someone tries to set overload higher than 100%
    (unless the specifically requested a percent value great than 100).
    
    Change-Id: Id030123153ea746671a8f1ca306d4b86e903fa22

commit 402a62eb5c3eafdfae8651f674f788ad21ce632e
Author: John Dickinson <email address hidden>
Date:   Tue Jan 27 20:50:33 2015 -0800

    2.2.2 changelog and authors update
    
    Change-Id: I8c18cfc2cd03242cf3c662494999f255cb7831c6

commit 77e926791665991aeb87ba1fef5d9e6ae6aff14e
Author: Jiangmiao Gao <email address hidden>
Date:   Wed Jan 28 22:12:57 2015 +0800

    Fix typing mistake for doc
    
    Change-Id: If3d4e4d44676ef987f584db8cb462ecaed03a019

commit 9b8dd0651d669ddd2922b6e8a4908a2f9e4b2663
Author: Pete Zaitcev <email address hidden>
Date:   Tue Jan 27 13:23:08 2015 -0700

    Don't set the already set Connection: close
    
    Meaningless cleanup of the day. In my defence, I hope the next guy
    doesn't have to grep throughout (the test is adjusted to signify).
    
    Change-Id: I9e10dd977d4ca48db1393519068ce0e286705433

commit 376dc5adc381f769091f29c75d9b26493383bac6
Author: Clay Gerrard <email address hidden>
Date:   Tue Jan 27 10:19:41 2015 -0800

    don't print cached dispersion if it's a lie
    
    Change-Id: I551fcaf274876861feb12848749590f220842d68

commit 3ac43e8299239657b37a3c52ef495d658b224a2e
Author: Samuel Merritt <email address hidden>
Date:   Thu Jan 22 15:26:19 2015 -0800

    Allow per-policy overrides in object replicator.
    
    The replicator already supports --devices and --partitions to restrict
    its operation to a subset of devices and partitions. However,
    operators don't always want to replicate a partition in all policies
    since different policies (usually) have different rings.
    
    For example, if I know that policy 0's partition 1234 is has no
    replicas on primary nodes due to over-aggressive rebalancing, I really
    want to find a node where the partition isa and make the replicator
    push it onto the primaries. However, if I haven't been messing with
    policy 1's ring, its partition 1234 is fine. With the existing
    replicator args, I get both or neither; this commit lets me get just
    the useful one.
    
    Change-Id: Ib1d58fdd228a6ee7865321e65d7c04a891fa5c49

commit f58f397cf6c2a33db21cabeab6c54163e503cde3
Author: Samuel Merritt <email address hidden>
Date:   Wed Jan 7 16:50:58 2015 -0800

    Make ThreadPools deallocatable.
    
    Currently, a ThreadPool acquires resources that last until process
    exit. You can let the ThreadPool go out of scope, but that doesn't
    terminate the worker threads or close file descriptors or anything.
    
    This commit makes it so you can .terminate() a ThreadPool object and
    get its resources back. Also, after you call .terminate(), trying to
    use the ThreadPool raises an exception so you know you've goofed.
    
    I have some internal code that could really use this, plus it makes
    the unit test run not leak resources, which is nice.
    
    Change-Id: Ibf7c6dc14c14f379421a79afb6c90a5e64b235fa

commit e5ca7e26a3beecaf149e7925658ae5f086b568c9
Author: Samuel Merritt <email address hidden>
Date:   Thu Jan 22 11:54:43 2015 -0800

    Optimize replication of targeted devices/partitions.
    
    swift-object-replicator lets you specify --devices and --partitions to
    perform a single replication pass over just those devices and
    partitions. However, it still scans every device and every partition
    to build up a list of jobs to do, then throws away the jobs for the
    wrong devices and partitions. This isn't too bad with partitions since
    it only wastes some CPU, but with devices, it results in unnecessary
    disk IO.
    
    This commit pushes the device and partition filtering a little further
    down into collect_jobs to avoid wasted work.
    
    Change-Id: Ia711bfc5a86ed4a080d27e08fe923cb4cb92da43

commit c6fde6de79278cbff7af243ae3bf538b88769fa9
Author: Mahati Chamarthy <email address hidden>
Date:   Mon Dec 8 23:28:48 2014 +0530

    Implement OPTIONS verb for storage nodes.
    
    Many times new deployers get mysterious errors after first setting up their
    Swift clusters. Most of the time, the errors are because the values in the ring
    are incorrect (e.g. a bad port number). OPTIONS will be used in a ring checker
    (which is WIP) that validates values in the ring.
    
    This patch includes OPTIONS for storage nodes and respective tests.
    
    Change-Id: Ia0033756d070bef11d921180e8d32a1ab2b88acf

commit 0bb690dca8f0f93bdccb6316d70c1c94a46c1b29
Author: Clay Gerrard <email address hidden>
Date:   Thu Jan 15 19:34:22 2015 -0800

    Exclude empty tiers from max_replicas
    
    If you have ring with less failure domains at a tier than replicas the
    max_replicas count for the failure domains at that tier already expect to have
    a max_replicas > 1.  For example, on a two node deployment, each of the two
    failure domains at the server tier is expected to have no more than *two*
    replicas of any part (surely it would be a problem if either server had all
    three replicas, but we expect some doubling).  If you have a third server, the
    max_replicas drops from 2 to 1 at each of those failure domains as they should
    only have at most a single copy of any partition or there's a dispersion
    problem.
    
    But if that third server is being decommissioned and all of it's device's
    weight are set to zero then you don't really have three failure domains at
    that tier and the max_replicas should be go back to 2 not 1.  Also any parts
    stuck in the tier with weight zero probably shouldn't be there - so you've got
    some kind of problem if there's *any* replicas in a zero weight tier
    *regardless* of the number of failure domains.
    
    I came to this when I was playing around with a topology migration that was
    moving a zone.  You start with two zones and everything is dispersed, you add
    a third zone and everything gets even *more* dispersed, then you remove one of
    the first zones that you're migrating off of and suddenly there's a problem
    because the two zones you're migrating to have > 1 replica of some parts.  Two
    zones was fine before, so what's the problem?
    
    Turns out the problem was a failure domain with zero weight still drains off a
    replica from the max_replicas of the other failure domains at that tier.
    
    Eventually I think it's pretty sane that the operator would remove the zero
    weight devices from the ring, and then max_replicas will go back to saying
    each of the two failure domains at this tier may have at most 2 replicas.  But
    because it's not always obvious when replication is finished just by looking
    at partition placement theres a disconnect and window where things seem to be
    saying it's worse than it really is.
    
    When a tier has zero weight max_replicas acts like the max_replicas at that
    tier should be 0.  We do expect that occasionally a tier with zero weight will
    have replicas assigned to it because min_part_hours - but I think thats also
    bad and worth flagging as a dispersion problem - although maybe it's
    conflating of seperate issues.
    
    Someone else might have a counter example where this would not be the desired
    behavior, and I'm not 100% sure it's going to always say the right thing in
    every use case - but as it's done currently I've got at least one concrete
    example where I think the current behavior doesn't make sense.
    
    Change-Id: I68d47ff0667788442342d55b8041aed80d0b2161

commit a0977ab8ca77a6ff899baa669134b242f328470a
Author: Prashanth Pai <email address hidden>
Date:   Mon Jan 12 12:51:46 2015 +0530

    dlo: Update doc about manifest containing data
    
    Current behavior:
    * If data/body is present in manifest file PUT request, the data/body gets
      saved onto disk, just like for a normal object.
    * Generally, this data in manifest file is never served on a GET response.
      However, when the manifest object path itself is part of prefix, GET
      response would contain data present in manifest file as well.
    * The query param multipart-manifest=get meant to retrieve SLO manifest
      also works in case of DLO manifest. Hence a COPY request with the
      multipart-manifest=get query param would actually copy DLO manifest.
    
    How things should have been:
    * The DLO manifest object is supposed to have no content and only have
      X-Object-Manifest metadata header.
    * Query param multipart-manifest=get is SLO specific and shouldn't have
      any role in DLO.
    
    This change intends to only document current behaviour and not change it,
    assuming there are users who have previously saved some content in DLO
    manifest file and/or have been using multipart-manifest=get to fetch
    and/or COPY the DLO manifest file with it's content.
    
    Change-Id: I0f6e175ad7752169ecf94df949336e0665928df7
    Signed-off-by: Prashanth Pai <email address hidden>

commit 20123399825c4ccacb78ebde56645856a6895a2f
Author: Clay Gerrard <email address hidden>
Date:   Thu Jan 8 20:29:47 2015 -0800

    Make more memcache options configurable
    
    More memcache options can be set in the memcache.conf or proxy-server.conf
    
     * connect_timeout
     * pool_timeout
     * tries
     * io_timeout
    
    Options set in proxy-server.conf are considered more specific to the memcache
    middleware.
    
    DocImpact
    
    Change-Id: I194d0f4d88c6cb8c797a37dcab48f2d8473e7a4e

commit 4ce4f98255989ad94cba126fc08a5bc2072a7c66
Author: Samuel Merritt <email address hidden>
Date:   Mon Jan 12 10:02:52 2015 -0800

    Drop redundant check in SLO segment-size validation
    
    Change-Id: Idf459f37cd18c46421c2e7a1a0506e8f28da13b4

commit a8bd2f737cd053db5cd9fb0926344acc1f68e2ce
Author: Clay Gerrard <email address hidden>
Date:   Tue Dec 30 00:22:44 2014 -0800

    Add dispersion command to swift-ring-builder
    
    Output a dispersion report that shows how many parts have each replica count
    at each tier along with some additional context.  Also the max_dispersion is a
    good canary for what a reasonable overload might be.
    
    Also display a warning on rebalance if the ring's dispersion is sub-optimal.
    
    The primitive form of the dispersion graph is cached on the builder, but the
    dispersion command will build it on the fly if you have a ring that was last
    rebalanced before the change.
    
    Also add --force option to rebalance to make it write a ring even if less than
    1% of parts moved.
    
    Try to clarify some dispersion and balance a little bit in the ring section of
    the architectural overview.
    
    Co-Authored-By: Christian Schwede <email address hidden>
    Co-Authored-By: Darrell Bishop <email address hidden>
    
    Change-Id: I7696df25d092fac56588080722e0a4167ed2c824

commit 3782bf1b562dbd61d933fd9031bcb56a6e674fb5
Author: Clay Gerrard <email address hidden>
Date:   Wed Jun 25 20:36:33 2014 -0700

    Remove the X-Newest pre-flight request on X-Timestamp
    
    There is a standard LBYL race that can better be addressed by making the
    EAFP case safer.
    
    Capture 409 Conflict during expect on PUT
    
    Similarly to how the proxy handles 412 on PUT, we will gather 409
    responses in the proxy during _connect_put_node.  Rather than skipping
    backend servers that already have a synced copy of an object we will
    accept their response and return 202 immediately.
    
    This is particularly useful to internal clients who are using
    X-Timestamp to sync transfers (e.g. container-sync and
    container-reconciler).
    
    No observable change in client facing behavior except that swift is
    faster to respond Accepted when it already has the data the client is
    purposing to send.
    
    Change-Id: Ie400d5bfd9ab28b290abce2e790889d78726095f

commit ba8114a513c17656696d3977292f464548f7a093
Author: Samuel Merritt <email address hidden>
Date:   Mon Dec 8 15:05:29 2014 -0800

    Improve object-replicator startup time.
    
    The object replicator checks each partition directory to ensure it's
    really a directory and not a zero-byte file. This was happening in
    collect_jobs(), which is the first thing that the object replicator
    does.
    
    The effect was that, at startup, the object-replicator process would
    list each "objects" or "objects-N" directory on each object device,
    then stat() every single thing in there. On devices with lots of
    partitions on them, this makes the replicator take a long time before
    it does anything useful.
    
    If you have a cluster with a too-high part_power plus some failing
    disks elsewhere, you can easily get thousands of partition directories
    on each disk. If you've got 36 disks per node, that turns into a very
    long wait for the object replicator to do anything. Worse yet, if you
    add in a configuration management system that pushes new rings every
    couple hours, the object replicator can spend the vast majority of its
    time collecting jobs, then only spend a short time doing useful work
    before the ring changes and it has to start all over again.
    
    This commit moves the stat() call (os.path.isfile) to the loop that
    processes jobs. In a complete pass, the total work done is about the
    same, but the replicator starts doing useful work much sooner.
    
    Change-Id: I5ed4cd09dde514ec7d1e74afe35feaab0cf28a10

commit 22b65846aa6d161ba963c9439b42fa4a9b7e1596
Author: Alistair Coles <email address hidden>
Date:   Mon Sep 8 16:54:41 2014 +0100

    Make probe tests tolerate deprecated policies
    
    A deprecated policy in swift.conf causes errors in
    probe tests that may attempt to use that policy.
    
    This patch introduces a list ENABLED_POLICIES in
    test/probe/common.py and changes probe tests to only
    use policies contained in that list.
    
    Change-Id: Ie65477c15d631fcfc3a4a5772fbe6d7d171b22b0