Bug #1487791 “POST to DLO squashes data without fast-POST” : Bugs : OpenStack Object Storage (swift)

Revision history for this message

Janie Richling (jrichli) wrote on 2015-12-04:

#1

from dfg in channel: if you send a COPY with --multipart-manifest=get on it then it just copies the manifest

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2015-12-09:

#2

So what *should* happen with a POST? (regardless of object_post_as_copy value - the outcome should be the same either way for a consistent API behaviour)

First, should a POST ever result in the DLO segments being "squashed" into what was the manifest object? IMHO, no - with post-as-copy that may result in errors when size of large object is > 5GB as reported.

Second, how should the presence or not of x-object-manifest header in the POST request be handled? I can see three options for the desired outcomes:

1) existing x-object-manifest value is *always preserved* on POST i.e. any x-object-manifest header sent with a POST is ignored, the original PUT x-object-manifest value can only be changed by another PUT.

2) existing x-object-manifest value is *always updated* with the value supplied with POST, and if POST has no x-object-manifest then the outcome is an object with no x-object-manifest value. i.e. client must provide x-object-manifest value with every POST in order to preserve the manifest property of the object.

3) existing x-object-manifest value is updated IF a new value is provided in the POST, otherwise the existing value is preserved. I strongly dislike this outcome because it is very hard to implement with object_post_as_copy=False (fast-POST). With fast-post it requires the object servers to maintain the timestamp of X-Object-Manifest values independently of the data or metadata timestamps in order to reconcile X-Object-Manifest values between replicas that may have become inconsistent. This is exactly the same problem that [1] is fixing with content-type, and it is non-trivial. So if we choose this option for post-as-copy we are adding significant pain to getting fast-post to provide the same behaviour.

So for me its between 1 and 2. Any other opinions?

[1] https://review.openstack.org/135380

Revision history for this message

John Dickinson (notmyname) wrote on 2015-12-09:

#3

option 1 sounds hard as soon as you add in multiple concurrent users.

I like option 2. That's the current behavior of metadata in objects. You set the current state of metadata on the POST. So basically, a POST to a *LO operates on the manifest, not the logical object. That's how I'd expect it to behave, and I think that's consistent with the way "normal" objects work.

Alistair Coles (alistair-coles) on 2015-12-10

Changed in swift:
status:	New → Confirmed

Revision history for this message

Eran Rom (eranr) wrote on 2015-12-10:

#4

Few thoughts:
1. Is there a reason to keep POST-as-COPY once FAST-post limitations are resolved? I know this is not really related to the above question, by the original bug description did brought it up.
2. To the question: Putting aside implementation difficulties (sorry) what should be the right semantics for x-object-manifest.
Of I get this right then a DLO is more of a metadata (over multiple objects) which happens to be implemented using a Swift object having some Swift (sys?) metadata over it. As such I think the x-object-manifest 'metadata' is essentially the DLO "data" and as such isn't supposed to be updated 'independently' of the Swift data object.
In other words, IMHO, x-object-manifest should not be up-datable via POST.
With this in mind, I would dare to suggest a 1* option which is: block any object POST request that has the 'x-object-manifest' header.

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2015-12-11:

#5

@John I'm not making an argument in favour of 1, but I'm not sure in what way 1 is hard with concurrent requests - basically x-object-manifest would get treated like Etag and x-object-sysmeta, can only be set on a PUT and the latest PUT timestamp will eventually win over other "concurrent" PUTs.

@Eran - interesting thought, whether we would just remove post-as-copy at same time as 'fixing' fast-POST. My assumption had been that there would at least be a period when both modes worked, to allow cautious operators to not 'upgrade' to fast-post.

Eran makes a good point about the semantics of x-object-manifest. With SLO, once a manifest has been PUT it can only cease to have the semantic of a manifest by being PUT again - POSTs to an SLO manifest will never change it from being a manifest, and the composition of the large object can only be changed by a PUT (since with SLO the segment references are in the object body).

So option 1 makes the semantic of a DLO manifest closer to that of an SLO (with the possible anomaly of making x-object-manifest the only item of client specified metadata that cannot be changed by a POST*, whereas option 2 is current behaviour with fast-post, but not post-as-copy because of the bug.

(* we'd also have to decide if we returned HttpBadRequest if x-object-manifest is sent with a POST, or simply ignore it :/)

Revision history for this message

Janie Richling (jrichli) wrote on 2016-01-05:

#6

In order to specify my vote on this, I would need to understand more what type of use cases would ever cause somebody to want to update the x-object-manifest.

Eran has said "x-object-manifest should not be up-datable via POST." I guess I want to make sure that this is correct and that it would never really make sense to update that value after the initial PUT. I mean, right now if somebody supplies the header, is it always with the same value that was there before anyway? I guess it might be to fix it if it was done incorrectly in the first place? If that is the only case, it is a failure case, so we can fall to deleting and re-creating.

Revision history for this message

Janie Richling (jrichli) wrote on 2016-01-07:

#7

It has been brought to my attention that there are use cases to update the x-object-mainfest. The manifest itself could be a segment. In which case, modifying the large object to be composed of different segments (re-using segments, including the manifest segment, in the storage) would lead to the desire to change this value.

In light of the fact that somebody may have been using this capability, I would vote for simply fixing the "bug" in this case, which is the situation initially described in this report. We can fix this behavior that is clearly wrong, and make the minimum amount of change to the current interface. I think that aligns with #2 at this point, but let me know if I am wrong. Hopefully I will have a patch loaded soon - either to the existing review or a new one. Let me know what you think.

Revision history for this message

Janie Richling (jrichli) wrote on 2016-01-27:

#8

I uploaded a patch that implements option #2: https://review.openstack.org/#/c/252791

Revision history for this message

clayg (clay-gerrard) wrote on 2016-01-27:

#9

wow, it's sorta tricky. I think as long as we don't make POST be semantically a COPY it's a win.

Whatever the behavior is just what the API is I suppose.

FWIW, not sure it's entirely fair to compare DLO semantics for update to SLO - since SLO's have a body and are validated - and DLO's can point to anything if you update the prefix.

I think most of the time POSTs to DLO manifests will be to add object/http metadata, it's trivial to change the DLO-prefix with a PUT if that's what you want to do, but OTOH I agree it would be inconsistent with other metadata in the namespace if a client request to change it ignored it.

However, from a API use-case perspective I think it *would* have been better to have it in sys-meta (exposed outside of user-meta namespace) and not updatable by anything except PUT. Not sure if we can fix that now, but that's MHO.

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2016-01-28:

#10

I don't think it would be too hard to make x-object-manifest behave like sysmeta, i.e. option 1 in #2 above, the header can only be set on a PUT, then is immutable. For fast-post we add it to the DATAFILE_SYSTEM_METADATA and...maybe its that simple, for post-as-copy have the copy-hook nuke the header from sink-req when handling a post-as-copy.

BUT...that feels more like an API change than option 2 when you remember that fast-post on master works and it works in the manner of option 2 i.e. it does not copy the segments into the manifest during POST (!), and it allows the header to be modified with a POST, and it=f the header is not specified on POST then it's lost from the object.

So anyone using fast-post (is there anyone?) could find the API changed for them if we do option #1 and make the header immutable, whereas anyone taking advantage of post-as-copy's wonky behavior to COPY during a POST can reasonably be told that was a bug.

Furthermore, if we change diskfile to make x-object-manifest header immutable then anyone who has previously updated a manifest with a fast-post would find that, after upgrading, their x-object-manifest values reverted back to whatever was PUT with the .data file. OOPS! So not only does the API change, but their effective data changes.

So although its not a clear cut choice, I'm for keeping the header as being mutable and ephemeral w.r.t. POSTs (IDK if ephemeral is the right word, transient perhaps?) i.e. option 2 in #2.

Revision history for this message

clayg (clay-gerrard) wrote on 2016-01-28:

#11

ok so when I said "Not sure if we can fix that now" I was probably being overly optimistic even entertaining the idea - I think Alister draws a clear argument that it obviously can *not* change it at this point.

To rephrase, I see three API's for POSTing DLO-prefix discussed here

1) fast-post - prefix is transient and updatable - consistent will all other user metadata
2) post-as-copy - totally broken
3) dreamy-eyed-time-machine - prefix is sysmeta

Now is not the time to invent a time machine.

We have a consistent behavior of POSTing DLO-prefix with fast-post and a broken implementation with post-as-copy. We should make the post-as-copy the same as fast-post. It's been said over and over - option #2 FTW.

Janie Richling (jrichli) on 2016-05-23

Changed in swift:
assignee:	nobody → Janie Richling (jrichli)

Revision history for this message

Janie Richling (jrichli) wrote on 2016-05-23:

#12

Now that copy has been moved to middleware, this issue - as written - no longer exists. But the tests that had been developed to assert the correct behavior from https://review.openstack.org/#/c/252791/20/test/functional/tests.py reveal that now when using post-as-copy and dlo, if you attempt a post of a new x-object-manifest header, it does not change the x-object-manifest. I do not yet know the solution. Maybe this should be written as a new bug that references this one.

Revision history for this message

Janie Richling (jrichli) wrote on 2016-07-26:

#13

The remaining issue mentioned in my last comment has now also been fixed on master. The only thing left to do is to add the dlo + post functests that were proposed in patch 252791. I will upload a new patch soon.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-07-26: Fix proposed to swift (master)

#14

Fix proposed to branch: master
Review: https://review.openstack.org/347545

Changed in swift:
status:	Confirmed → In Progress

Janie Richling (jrichli) on 2016-07-26

summary:

- POST to DLO squashes data without fast-POST
+ No tests exist for POST to DLO manifest file

Revision history for this message

Janie Richling (jrichli) wrote on 2016-07-27: Re: No tests exist for POST to DLO manifest file

#15

Well, technically, the behavior that exists on master currently matches what was described in "option #1" (thus only a test would need to be added, but no actual DLO changes). Perhaps it is worth re-assessing whether or not we need to make changes to behave as described in option #2. Is it important to allow the client to modify the x_o_m on an existing DLO manifest file?

Revision history for this message

Tim Burke (1-tim-z) wrote on 2016-07-27:

#16

Current state of the world:

* With post-as-copy, POSTing to a DLO preserves the original X-Object-Manifest, whether the client sent an X-Object-Manifest or not.
* With fast-post, POSTing an X-Object-Manifest to a DLO updates the manifest.
* With fast-post, POSTing to a DLO while not providing an X-Object-Manifest turns it into a plain old swift object (!). Thought you were setting this to expire next week? Well, it's (effectively) already gone.
* With either, POSTing an X-Object-Manifest to a plain old swift object turns it into a DLO (!). This means that for post-as-copy the only way to turn it back into a POSO is to re-upload (!!).
* With either, POSTing an X-Object-Manifest to an SLO results in ... poorly defined behavior [1]

I'd prefer to have the manifest header be treated like sysmeta. Sure, it's user meta in the sense that the user tells us the value, but it seems like a much cleaner semantic to say (1) you can only use *one* type of large object at a time and (2) whatever type of large object you use, the fact that it is a large object and all of the manifest data is set on PUT. We could enforce this with something like what Clay's trying to do for SLO in https://review.openstack.org/#/c/334719/. This also matches fairly well the behavior we're aiming for with symlinks.

But Alistair's right. I'm less worried about the degree to which the API changes -- I'm comfortable with claiming that the old behavior was a bug that is now fixed. Dealing with on-disk data where a user has modified the manifest via a POST, however, is rather terrible. If we modeled enforcement off of Clay's SLO patch, we change the effective data immediately on upgrade. Maybe we could get away with something like the *current* SLO behavior? I.e., prevent POSTs from including X-Object-Manifest at the proxy, then carry forward any existing X-Object-Manifest entries manually. (Maybe only if orig_metadata came from a .meta instead of a .data?) But we'd still run into consistency issues when we have a PUT over a modified DLO followed by a POST.

[1] My manually testing resulted in 500s, but that seems to have been caused (in part) by my test data: the reconstructed DLO was all integer data, which could be parsed as JSON, resulting in a TypeError around `for seg_dict in segments`. This is pipeline dependent, though; I had slo left of dlo, and by switching the order the object would behave as a DLO. At least, until I broke the SLO, at which point it would 409.

Current state of the world:

* With post-as-copy, POSTing to a DLO preserves the original X-Object-Manifest, whether the client sent an X-Object-Manifest or not.
 * With fast-post, POSTing an X-Object-Manifest to a DLO updates the manifest.
 * With fast-post, POSTing to a DLO while not providing an X-Object-Manifest turns it into a plain old swift object (!). Thought you were setting this to expire next week? Well, it's (effectively) already gone.
 * With either, POSTing an X-Object-Manifest to a plain old swift object turns it into a DLO (!). This means that for post-as-copy the only way to turn it back into a POSO is to re-upload (!!).
 * With either, POSTing an X-Object-Manifest to an SLO results in ... poorly defined behavior [1]

I'd prefer to have the manifest header be treated like sysmeta. Sure, it's user meta in the sense that the user tells us the value, but it seems like a much cleaner semantic to say (1) you can only use *one* type of large object at a time and (2) whatever type of large object you use, the fact that it is a large object and all of the manifest data is set on PUT. We could enforce this with something like what Clay's trying to do for SLO in https://review.openstack.org/#/c/334719/. This also matches fairly well the behavior we're aiming for with symlinks.

But Alistair's right. I'm less worried about the degree to which the API changes -- I'm comfortable with claiming that the old behavior was a bug that is now fixed. Dealing with on-disk data where a user has modified the manifest via a POST, however, is rather terrible. If we modeled enforcement off of Clay's SLO patch, we change the effective data immediately on upgrade. Maybe we could get away with something like the *current* SLO behavior? I.e., prevent POSTs from including X-Object-Manifest at the proxy, then carry forward any existing X-Object-Manifest entries manually. (Maybe only if orig_metadata came from a .meta instead of a .data?) But we'd still run into consistency issues when we have a PUT over a modified DLO followed by a POST.

[1] My manually testing resulted in 500s, but that seems to have been caused (in part) by my test data: the reconstructed DLO was all integer data, which could be parsed as JSON, resulting in a TypeError around `for seg_dict in segments`. This is pipeline dependent, though; I had slo left of dlo, and by switching the order the object would behave as a DLO. At least, until I broke the SLO, at which point it would 409.

Revision history for this message

Janie Richling (jrichli) wrote on 2016-07-28:

#17

Thank you for the details, Tim. Given clay's SLO patch and current DLO behavior, it sounds like we should not allow POST to change where a manifest points. Is this also what you are thinking? I am thinking about writing a new bug to handle the undesirable behavior that currently happens when X-Object-Manifest is specified on a POST to a plain object (that patch could be dependent on clay's SLO patch).

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2016-08-01:

#18

Download full text (3.6 KiB)

So somehow we went from POST-as-copy to a DLO manifest causing it to be replaced with the full object, to POST to a DLO manifest not updating the manifest header :/ That in itself confirms the current topic of this bug.

I am wondering if we should have closed the old bug and started a new one?

Re. changing fast-post behaviour: I cannot (yet) think of how we can "safely" make X-Object-Manifest become an immutable property of an object without risking breaking existing data. More discussion on that below.

So my conclusion is unchanged: however much I like or dislike the API, I think that post-as-copy should be made to behave the same as fast-post, which at this moment means that it is a bug for the post-as-copy to not update the x-object-manifest header. i.e. I still end up voting for option 2 described in comment #2.

*** objects that are both DLO and SLO

As for objects that are both DLO and SLO, maybe we should declare DLO to be preferred i.e. SLO GET/HEAD handling is bypassed when X-Object-Manifest is found in response headers. I *think* that avoids problems whichever way the middlewares are ordered.

If there is a pre-existing object with both headers that was supposed to be GETtable as an SLO then (a) did it ever work? and (b) you can make it an SLO by removing the X-Object-Manifest header with a POST (subject to fixing the post-as-copy to a DLO manifest).

Can a DLO/SLO currently ever work for GETs? Let's say you had a SLO manifest at a/c1/m1, you then added X-Object-Manifest=a/c2 with a fast-post (remember, you cannot yet do that with post-as-copy) and the x-object-manifest resolves to a different valid SLO manifest a/c2/m2. Currently with SLO to left of DLO (which the client has no control over) a GET for the SLO manifest at a/c/m1 would result in DLO first resolving a/c/m2 and passing its body to SLO, which would SLO would then successfully parse that but probably find an ETag mismatch wrt to the headers on a/c/m1 (I forget which headers DLO would return with its response...).

Maybe if the SLO manifest st a/c/m1 were identical to that at a/c/m2 then the GET would work on master?

*** Why it is hard to make x-object-manifest become immutable ***

Consider a pre-existing object:

t1.data{x-object-manifest=bar} + t2.meta{x-object-manifest=foo}

we'd have to make the new algorithm favour foo over bar i.e. preserve the .meta file value, so we could copy it from old metadata to new metadata (maybe what Tim means by "carry forward...manually "). But now consider a pre-existing object:

t1.data{x-object-manifest=bar} + t2.meta{}

there is nothing to carry forward from the .meta file, but we must not carry-forward the data file value (bar).

Now consider a post-changed-semantic POST to an object:

t1.data{x-object-manifest=bar} + t3.meta{}

...now we *do* want to carry-forward the .data file value.

We could try to be clever and have the object server write a marker to inform the future reader that the .meta file was written in the post-changed-semantic future world:

t1.data{x-object-manifest=bar} + t3.meta{x-object-sysmeta-use-the-datafile-object-manifest=True}

but that violates the rpinciple that you should never change the perceive...

So somehow we went from POST-as-copy to a DLO manifest causing it to be replaced with the full object, to POST to a DLO manifest not updating the manifest header :/ That in itself confirms the current topic of this bug.

I am wondering if we should have closed the old bug and started a new one?

Re. changing fast-post behaviour: I cannot (yet) think  of how we can "safely" make X-Object-Manifest become an immutable property of an object without risking breaking existing data. More discussion on that below.

So my conclusion is unchanged: however much I like or dislike the API, I think that post-as-copy should be made to behave the same as fast-post, which at this moment means that it is a bug for the post-as-copy to not update the x-object-manifest header. i.e. I still end up voting for option 2 described in comment #2.

*** objects that are both DLO and SLO

As for objects that are both DLO and SLO, maybe we should declare DLO to be preferred i.e. SLO GET/HEAD handling is bypassed when X-Object-Manifest is found in response headers. I *think* that avoids problems whichever way the middlewares are ordered.

If there is a pre-existing object with both headers that was supposed to be GETtable as an SLO then (a) did it ever work? and (b) you can make it an SLO by removing the X-Object-Manifest header with a POST (subject to fixing the post-as-copy to a DLO manifest).

Can a DLO/SLO currently ever work for GETs? Let's say you had a SLO manifest at a/c1/m1, you then added X-Object-Manifest=a/c2 with a fast-post (remember, you cannot yet do that with post-as-copy) and the x-object-manifest resolves to a different valid SLO manifest a/c2/m2. Currently with SLO to left of DLO (which the client has no control over) a GET for the SLO manifest at a/c/m1 would result in DLO first resolving a/c/m2 and passing its body to SLO, which would SLO would then successfully parse that but probably find an ETag mismatch wrt to the headers on a/c/m1 (I forget which headers DLO would return with its response...).

Maybe if the SLO manifest st a/c/m1 were identical to that at a/c/m2 then the GET would work on master?

*** Why it is hard to make x-object-manifest become immutable ***

Consider a pre-existing object:

t1.data{x-object-manifest=bar} + t2.meta{x-object-manifest=foo}

we'd have to make the new algorithm favour foo over bar i.e. preserve the .meta file value, so we could copy it from old metadata to new metadata (maybe what Tim means by "carry forward...manually "). But now consider a pre-existing object:

t1.data{x-object-manifest=bar} + t2.meta{}

there is nothing to carry forward from the .meta file, but we must not carry-forward the data file value (bar).

Now consider a post-changed-semantic POST to an object:

t1.data{x-object-manifest=bar} + t3.meta{}

...now we *do* want to carry-forward the .data file value.

We could try to be clever and have the object server write a marker to inform the future reader that the .meta file was written in the post-changed-semantic future world:

t1.data{x-object-manifest=bar} + t3.meta{x-object-sysmeta-use-the-datafile-object-manifest=True}

but that violates the rpinciple that you should never change the perceived timestamp of an item of state - in this case x-object-sysmeta-use-the-datafile-object-manifest=True causes the perceived value of x-object-manifest at t3 to be 'bar' which was actually set at t1. On another object server (where maybe our t2.meta file exists) a different outcome would be reached at t3 and we have an inconsistency in t3.meta that isn't resolved by current mechanisms.

but that doesn't work if on another object server the POST at t3 happened upon t2.meta

Revision history for this message

Janie Richling (jrichli) wrote on 2016-08-02:

#19

Thanks, Alistair.

How about this for a way to proceed:
We have one patch that 'partially' fixes this bug - which simply adds the test that asserts if the x_o_m is provided, then POST to DLO does not squash the contents.

Have a different patch to add documentation and also add the code to allow post-as-copy to change a dlo manifest on a post. We can also resolve what we want to do with SLO referencing DLO or vice-versa in this patch.

I can create a separate bug for the second patch described here, if that is best.

This way, the dlo test gets in place as soon as possible - and I will work on supporting option #2 and make proper docs when the issues are settled.

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2016-08-04:

#20

@Janie - OK, adding test for current behaviour would be good for as much as the behaviour is common to fast-post and post-as-copy i.e. not squashing the manifest. Thanks.

Maybe revert this bug to original topic and we'll close it (do we know the commit that fixed the original bug - was it copy middleware?). Start a new bug for poast-as-copy not allowing manifest to be updated.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-08-10: Change abandoned on swift (master)

#21

Change abandoned by Alistair Coles (<email address hidden>) on branch: master
Review: https://review.openstack.org/252791
Reason: The fix is out of date w.r.t. copy feature being moved to copy middleware [1], at which point I believe the bug was fixed.

Apparently the useful tests here have been included in [2]

Please restore the review if I am incorrect.

[1] https://review.openstack.org/156923
[2] https://review.openstack.org/#/c/347545/

Janie Richling (jrichli) on 2016-08-13

summary:

- No tests exist for POST to DLO manifest file
+ POST to DLO squashes data without fast-POST

Revision history for this message

Janie Richling (jrichli) wrote on 2016-08-13:

#22

I verified that it was copy as middleware that fixed the original issue. (The segment data was appended to the manifest contents)
FIXED : commit 46d61a4dcd9a5d9157625c06d6fe7d916e80c3d2 Refactor server side copy as middleware
HAS ISSUE: commit 72372c1464d1aae4a9b5de5ef6a3689fddf168cc Merge "decouple versioned writes from COPY"

What is interesting about the old bug is that if the manifest header was supplied on each POST (post-as-copy), then it is clear that the segment data was getting *appended* to the manifest contents (not replacing its contents), so the manifest object would grow by the size of the large object with each post. The appended object data was revealed with every subsequent large object GET after the POST (since the manifest contents are always included first before the segment data).

If the manifest header was NOT supplied on this POST, then the manifest was no longer a manifest, and the contents that had been appended remain the contents (it doesn't grow with subsequent POSTs only because the manifest-ness was lost).

There are still no tests that POST to a DLO manifest, but that is being worked under https://review.openstack.org/#/c/347545/ with no associated bug (Alistair mentioned we could close this as fixed under copy as mw).

Bug https://bugs.launchpad.net/swift/+bug/1612991 was created for the "Post-as-copy does not allow a DLO manifest to be updated".

Revision history for this message

Alistair Coles (alistair-coles) wrote on 2016-08-16:

#23

IMO this bug is fixed, as Janie confirmed the fix was in this commit 46d61a4. So this bug could be closed.

Alistair Coles (alistair-coles) on 2016-08-16

Changed in swift:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-08-18: Related fix merged to swift (master)

#24

Reviewed: https://review.openstack.org/347545
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=13747021a8743d358961c1c4820fddfe9c955ea2
Submitter: Jenkins
Branch: master

commit 13747021a8743d358961c1c4820fddfe9c955ea2
Author: Janie Richling <email address hidden>
Date: Tue Jul 26 15:20:52 2016 -0500

Add test for POST to DLO manifest file

    In the past, a POST to a DLO manifest file when object_post_as_copy
    was true resulted in the manifest file contents being replaced
    by the concatenation of the DLO segments. This no longer
    happens, but tests for this case are missing.

This patch adds a functional test to assert that the manifest
file is preserved in a POST request.

    Change-Id: I90546014a7dcc7266f0d0e0ff6339688b7954b96
    Related-bug: #1487791
    Related-bug: #1514317

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-08-30: Related fix proposed to swift (feature/hummingbird)

#25

Related fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/363111

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-08-30: Related fix merged to swift (feature/hummingbird)

#26

Download full text (84.1 KiB)

Reviewed: https://review.openstack.org/363111
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=1ab2a296f58ae76aeffef9f3f0fb90e15358be27
Submitter: Jenkins
Branch: feature/hummingbird

commit 3b5850836c59c46f2507a7f62aceccf4c37e5d41
Author: gecong1973 <email address hidden>
Date: Tue Aug 30 15:08:49 2016 +0800

Remove white space between print and ()

There is a white space between print and ()
in /tempauth.py, This patch fix it

Change-Id: Id3493bdef12223aa3a2bffc200db8710f5949101

commit f88e7fc0da2ed6a63e0ea3c3459d80772b3068cd
Author: zheng yin <email address hidden>
Date: Mon Aug 29 20:21:44 2016 +0800

Clarify test case in common/ring/test_builder

    They use a bare assertRaises(ValueError, ring.RingBuilder, *,*,*), but
    it's not clear which one raises which ValueError(), so I extend them to
    validate the error strings as well.

Change-Id: I63280a9fc47ff678fe143e635046a0b402fd4506

commit d68b1bd6ddf44c5088e9d02dcb2f1b802c71411b
Author: zhufl <email address hidden>
Date: Mon Aug 29 14:31:27 2016 +0800

Remove unnecessary tearDown

This is to remove unnecessary tearDown to keep code clean.

Change-Id: Ie70e40d6b55f379b0cc9bc372a35705462cade8b

commit d2fc2614575b04fd9cab5ae589880b92eee9b186
Author: Matthew Oliver <email address hidden>
Date: Fri Aug 19 16:17:31 2016 +1000

Authorise versioned write PUTs before copy

    Currently a versioned write PUT uses a pre-authed request to move
    it into the versioned container before checking whether the
    user is authorised. This can lead to some interesting behaviour
    whereby a user can select a versioned object path that it does not
    have access to, request a put on that versioned object, and this
    request will execute the copy part of the request before it fails
    due to lack of permissions.

This patch changes the behaviour to be the same as versioned DELETE
where the request is authorised before anything is moved.

Change-Id: Ia8b92251718d10b1eb44a456f28d3d2569a30003
Closes-Bug: #1562175

commit c1ef6539b6ba9d2e4354c9cd2eec8a0195cdb19f
Author: Clay Gerrard <email address hidden>
Date: Thu Aug 25 11:00:49 2016 -0700

add test for expirer processes == process

This is a follow up from a change that improved the error message.

Related-Change: I3d12b79470d122b2114f9ee486b15d381f290f95

Change-Id: I093801f3516a60b298c13e2aa026c11c68a63792

commit 01477c78c1163822de41484e914a0736e622085b
Author: zheng yin <email address hidden>
Date: Thu Aug 25 15:37:42 2016 +0800

Fix ValueError information in obj/expirer

    I fix error information in raise ValueError(...)
    For example:
        if a>=b:
            # It should be under below and not 'a must be less than or equal to b'
            raise ValueError('a must be less than b')

Change-Id: I3d12b79470d122b2114f9ee486b15d381f290f95

commit b81f53b964fdb8f3b50dd369ce2e194ee4dbb0b7
Author: zheng yin <email address hidden>
Date: Tue Aug 23 14:26:47 2016 +0800

Improve readability in the obj server's unit tests

This change improves the reada...

Reviewed:  https://review.openstack.org/363111
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=1ab2a296f58ae76aeffef9f3f0fb90e15358be27
Submitter: Jenkins
Branch:    feature/hummingbird

commit 3b5850836c59c46f2507a7f62aceccf4c37e5d41
Author: gecong1973 <ge.cong@zte.com.cn>
Date:   Tue Aug 30 15:08:49 2016 +0800

Remove white space between print and ()
    
    There is a white space between print and ()
    in /tempauth.py, This patch fix it
    
    Change-Id: Id3493bdef12223aa3a2bffc200db8710f5949101

commit f88e7fc0da2ed6a63e0ea3c3459d80772b3068cd
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Mon Aug 29 20:21:44 2016 +0800

Clarify test case in common/ring/test_builder
    
    They use a bare assertRaises(ValueError, ring.RingBuilder, *,*,*), but
    it's not clear which one raises which ValueError(), so I extend them to
    validate the error strings as well.
    
    Change-Id: I63280a9fc47ff678fe143e635046a0b402fd4506

commit d68b1bd6ddf44c5088e9d02dcb2f1b802c71411b
Author: zhufl <zhu.fanglei@zte.com.cn>
Date:   Mon Aug 29 14:31:27 2016 +0800

Remove unnecessary tearDown
    
    This is to remove unnecessary tearDown to keep code clean.
    
    Change-Id: Ie70e40d6b55f379b0cc9bc372a35705462cade8b

commit d2fc2614575b04fd9cab5ae589880b92eee9b186
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Fri Aug 19 16:17:31 2016 +1000

Authorise versioned write PUTs before copy
    
    Currently a versioned write PUT uses a pre-authed request to move
    it into the versioned container before checking whether the
    user is authorised. This can lead to some interesting behaviour
    whereby a user can select a versioned object path that it does not
    have access to, request a put on that versioned object, and this
    request will execute the copy part of the request before it fails
    due to lack of permissions.
    
    This patch changes the behaviour to be the same as versioned DELETE
    where the request is authorised before anything is moved.
    
    Change-Id: Ia8b92251718d10b1eb44a456f28d3d2569a30003
    Closes-Bug: #1562175

commit c1ef6539b6ba9d2e4354c9cd2eec8a0195cdb19f
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Aug 25 11:00:49 2016 -0700

add test for expirer processes == process
    
    This is a follow up from a change that improved the error message.
    
    Related-Change: I3d12b79470d122b2114f9ee486b15d381f290f95
    
    Change-Id: I093801f3516a60b298c13e2aa026c11c68a63792

commit 01477c78c1163822de41484e914a0736e622085b
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Thu Aug 25 15:37:42 2016 +0800

Fix ValueError information in obj/expirer
    
    I fix error information in raise ValueError(...)
    For example:
        if a>=b:
            # It should be under below and not 'a must be less than or equal to b'
            raise ValueError('a must be less than b')
    
    Change-Id: I3d12b79470d122b2114f9ee486b15d381f290f95

commit b81f53b964fdb8f3b50dd369ce2e194ee4dbb0b7
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Tue Aug 23 14:26:47 2016 +0800

Improve readability in the obj server's unit tests
    
    This change improves the readability of the object-server's unit
    tests by breaking down some long assertTrue statements into smaller
    and much easier to read and more relevant assert statements.
    
    For example:
        assertTrue(a in resp.headers and
                   b in resp.headers and
                   c not in resp.headers)
    
    Is equal to:
        assertIn(a, resp.headers)
        assertIn(b, resp.headers)
        assertNotIn(c, resp.headers)
    
    Change-Id: Iba746ecfb1a1dc541856b7a4c9d2f00d08e4ad51

commit 89388bf232b54ec0adf5cb815efff23cd479e2c1
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Mon Aug 22 19:27:35 2016 +0000

Fix typos and grammer in builder.py
    
    Change-Id: Ib87f4df8f741809840e92db9bacf2af847a5f77f
    Closes-Bug: #1600403

commit d98928caa99cbfac1b8710ac0761925a8146422e
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Aug 22 10:41:59 2016 +0200

py3: tox.ini: use substituation to py35 commands
    
    Use tox substitution to avoid duplicating testenv:py34 commands in
    testenv:py35, to not have to maintain the whitelist of Python 3 tests
    in two different places.
    
    Write also the list of tests in a newline to be able to more easily
    add new unit tests.
    
    Change-Id: I6e7f238f1c5d3fc9b6560918dcbb93e9dd8ec084

commit aa893d90778f912fa6668f9e3c462885654c423a
Author: Graham Hayes <graham.hayes@hpe.com>
Date:   Fri Aug 19 14:25:06 2016 +0100

Get ready for os-api-ref sphinx theme change
    
    Change-Id: Ib4aa4a26814273efafa3453237d18acf8cc966cb

commit 8bf2233b40dbf2321f754b2a08e8271b4c0ee007
Author: Peter Lisák <peter.lisak@firma.seznam.cz>
Date:   Thu Aug 18 16:14:36 2016 +0200

Documantation enhancements of nice/ionice feature
    
    Based on comments from patch #238799.
    
    Change-Id: I9455cf6dc7fd12fee62439ff3c5f3255287ab1be

commit 13747021a8743d358961c1c4820fddfe9c955ea2
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Tue Jul 26 15:20:52 2016 -0500

Add test for POST to DLO manifest file
    
    In the past, a POST to a DLO manifest file when object_post_as_copy
    was true resulted in the manifest file contents being replaced
    by the concatenation of the DLO segments.  This no longer
    happens, but tests for this case are missing.
    
    This patch adds a functional test to assert that the manifest
    file is preserved in a POST request.
    
    Change-Id: I90546014a7dcc7266f0d0e0ff6339688b7954b96
    Related-bug: #1487791
    Related-bug: #1514317

commit b7b77c7aa3aea2d210f77b0da4b69e6bdec49c84
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Aug 15 17:52:58 2016 -0700

Follow up delayed reap probe test
    
    This is a follow up patch for https://review.openstack.org/#/c/321041
    
    This patch includes following items:
    
    - Move test_sync to below _verify_account_reaped to be the actual tests
      as a bunch
    - Change the test name fron "test_sync" to "test_reap" to clarify the
      purpose
    - Fix a typo from "Object" to "Container" for an error message
    
    Change-Id: I51bc01113056e2eb99f731d38e9f1c7a6c5c96be

commit 26633af3af9a1998b1e9e3fdf061482255329571
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Thu Aug 11 20:44:37 2016 +0800

Reset AccountReaper stats in __init__
    
    Make the AccountReaper __init__ method reset its stats variables.
    This saves the unit test having to initialise the stats variables.
    Also add more asserts to some AccountReaper test cases.
    
    Change-Id: Iea112962d89ebfa3450f43b2a28ac8e8ed8b07b0

commit ce49a296c6f3e0bdeae0c3cd6615d31141f5c23e
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Aug 17 17:20:21 2016 +0100

Add rm to tox whitelist_externals
    
    Avoids a warning when running tox -e api-ref
    
    Change-Id: Ib02849075e6424f1db84499fd7500d7bb76dde67

commit dfa5523d8c729dd44f79015d8ad3d0260e7f86c3
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Apr 11 20:51:00 2016 -0700

Add Pros/Cons docs for global cluster consideration
    
    This comes from discussion in Bristol Hackathon (Feb 2016).
    Currently Swift has a couple of choices (Global Cluster and Container
    Sync) to sync the stored data into geographically distributed locations.
    
    This patch adds the summary of the discussion comparing between
    Global Cluster and Container Sync to enable operators to know which
    functionality fits their own use case.
    
    And, to be fairness with container-sync, this patch moves global
    cluster docs into overview_global_cluster.rst from admin_guide.rst.
    
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    
    Change-Id: I624eb519503ae71dbc82245c33dab6e8637d0f8b

commit 95a5a4a7ec1ec8a48db756121e7b3440f5704536
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Aug 15 18:26:00 2016 -0700

Don't run probe tests if resetswift failed
    
    Probe test is cleaning up the swift environment for each test in setUp
    method. However, probe tests will run even if we cannot use the resetswift
    script for some reasons (e.g. not permitted, the script not found) and
    probably the probe tests will fail after a long time passed for the
    execution.
    
    To prevent such an unfortunate situation and also to find the reason
    easily, this patch adds the exit code check for "resetswift" and if it
    failed, the test will raise AssertionError with the stdout and stderr to
    make it easy to find the reason.
    
    Closes-Bug: #1613494
    
    Change-Id: Id80d56ab6b71402ead4fe22c120064d78c1e74ac

commit 30b97f2367ed078a7fe278c79518cd6d8c1162b0
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Apr 19 09:41:50 2016 -0700

Drop X-Auth-Token from all versioned_writes subrequests
    
    It is not necessary for versioned_writes to function (all of these
    were pre-authed requests anyway), and transaction ID should be used to
    trace requests instead.
    
    Change-Id: If55c1586aa38f9a3bc9e1d00768ca00201af94cd

commit cc2b2cf9c8a5cb913817e2f900c676809cd2e027
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Aug 16 16:21:59 2016 +0100

Improve doc for using container-sync with large objects
    
    Clarify that synced segment container names must be the same
    when syncing large objects.
    
    Also add multipart-menifest query string option to API ref
    for object GETs.
    
    Change-Id: Ib2d2a1e6c1e5eff215fc75c2b49e7d6758b17b7e
    Partial-Bug: #1613681
    Closes-Bug: #1613316

commit 65b1820407ea40bd7d65a5356a58a689befe3cb5
Author: Charles Hsu <charles0126@gmail.com>
Date:   Thu Aug 11 00:53:13 2016 +0800

Ignore auditor status files to prevent replicator reports errors
    
    Ignore `auditor_status_*.json` files during the collecting jobs
    and replicator won't use these wrong paths to find objects that
    causes an exception to increase failure count in replicator report.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Co-Authored-By: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
    
    Change-Id: Ib15a0987288d9ee32432c1998aefe638ca3b223b
    Closes-Bug: #1583305

commit c7283be4fe9ebaf0e8feba695a1c664eec6fb355
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Aug 19 12:17:47 2015 -0700

Add "history" mode to versioned_writes middleware
    
    This change introduces the concept of a "versioning mode" for
    versioned_writes. The following modes are supported:
    
     * stack
    
        When deleting, check whether any previous versions exist in the
        versions container. If none is found, the object is deleted. If the
        most-recent version in the versions container is not a delete
        marker, it is copied into the versioned container (overwriting the
        current version if one exists) and then deleted from the versions
        container. This preserves the previous behavior.
    
        If the most-recent version in the versions container is a delete
        marker and a current version exists in the versioned container, the
        current version is deleted. If the most-recent version in the
        versions container is a delete marker and no current version exists
        in the versioned container, we copy the next-most-recent version
        from the versions container into the versioned container (assuming
        it exists and is not a delete marker) and delete both the
        most-recent version (i.e., the delete marker) and the just-copied
        next-most-recent version from the versions container.
    
        With this mode, DELETEs to versioned containers "undo" operations
        on containers. Previously this was limited to undoing PUTs, but now
        it will also undo DELETEs performed while in "history" mode.
    
     * history
    
        When deleting, check whether a current version exists in the
        versioned container. If one is found, it is copied to the versions
        container. Then an empty "delete marker" object is also put into the
        versions container; this records when the object was deleted.
        Finally, the original current version is deleted from the versioned
        container. As a result, subsequent GETs or HEADs will return a 404,
        and container listings for the versioned container do not include
        the object.
    
        With this mode, DELETEs to versioned containers behave like DELETEs
        to other containers, but with a history of what has happened.
    
    Clients may specify (via a new X-Versions-Mode header) which mode a
    container should use. By default, the existing "stack" mode is used.
    
    Upgrade consideration:
    ======================
    
    Clients should not use the "history" mode until all proxies in the
    cluster have been upgraded. Attempting to use the "history" mode during
    a rolling upgrade may result in some requests being served by proxies
    running old code (which necessarily uses the "stack" mode), leading to
    data loss.
    
    Change-Id: I555dc17fefd0aa9ade681aa156da24e018ebe74b

commit bb87fcefced099a56537edd0db7ca965e9902f5a
Author: liujiong <liujiong@gohighsec.com>
Date:   Sun Aug 14 00:43:47 2016 +0800

Fix typo in the file
    
    Change-Id: I6539e9b9fb7918e387e8ae802be7b4efbcb07f4d

commit aab2cee827b18fe1c48f121d2c9066af2dd0c366
Author: Andreas Jaeger <aj@suse.com>
Date:   Fri Aug 12 21:18:07 2016 +0200

Move other-requirements.txt to bindep.txt
    
    The default filename for documenting binary dependencies has been
    changed from "other-requirements.txt" to "bindep.txt" with the release
    of bindep 2.1.0. While the previous name is still supported, it will
    be deprecated.
    
    Move the file around to follow this change.
    
    Note that this change is self-testing, the OpenStack CI infrastructure
    will use a "bindep.txt" file to setup nodes for testing.
    
    For more information about bindep, see also:
    http://docs.openstack.org/infra/manual/drivers.html#package-requirements
    http://docs.openstack.org/infra/bindep/
    
    As well as this announcement:
    http://lists.openstack.org/pipermail/openstack-dev/2016-August/101590.html
    
    Change-Id: I000a4e708006263acc6d9731a6677a6e62e285b6

commit f7a820ed3a72992948e22e7366a9a1780ad40388
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Aug 12 05:46:33 2016 +0000

Wait for a non-empty chunk in WSGIContext._app_call
    
    We're functioning as a WSGI server here, so this bit from PEP-3333 seems
    to apply:
    
    > The start_response callable must not actually transmit the response
    > headers. Instead, it must store them for the server or gateway to
    > transmit only after the first iteration of the application return
    > value that yields a non-empty bytestrin ... . In other words, response
    > headers must not be sent until there is actual body data available, or
    > until the application's returned iterable is exhausted.
    
    Plus, it mirrors what swob.Request.call_application does.
    
    Change-Id: I1e8501f8ce91ea912780db64fee1c56bef809a98

commit 05b8d9d7fc4e5d1c64385c157431215acda38852
Author: Thiago da Silva <thiago@redhat.com>
Date:   Thu Aug 11 16:24:57 2016 -0400

fix swift_oldies on RH based systems
    
    Change-Id: Icbb4c6d461ded4fab2afade09e718b3a74917717
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit 7e2cb23f887cf0092e0bd9f2baa42d3ddbb257ce
Author: Nakul Dahiwade <nakul.dahiwade@intel.com>
Date:   Thu Aug 11 16:10:35 2016 +0000

Grammer error : swift/doc/source/overview_ring.rst
    
    Changed sentence: "Regions can be used to describe geo-graphically
    systems characterized by lower-bandwidth"
    To: "Regions can be used to describe geographical
    systems characterized by lower-bandwidth"
    
    Change-Id: I0f614a4c53dd31459f1b6297dd32a8c0f609d9ce
    Closes-Bug: 1612302

commit 844d0c965df1fcec53f118ed71c2a560cfe88a2b
Author: Shashank Kumar Shankar <shashank.kumar.shankar@intel.com>
Date:   Thu Aug 11 05:21:54 2016 +0000

Corrects spelling error in swift/common/middleware/slo.py
    
    Fixes 'perfomed' to 'performed'
    
    Change-Id: I54adf45494cd4c6edae7bb5b404d377527c6c5a0
    Closes-Bug: 1612051

commit 4638171ece4e70408f5b897a9bd1c12a70696aa4
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Wed Aug 10 16:48:45 2016 +0800

Fix assertEqual error
    
    I think assertEqual(a,a) is error that caused by carelessness
    in test case, so I modify it.
    
    Change-Id: I8767c35e8a1a47f1b64241f67959277074a37b21

commit ed772236c7289d178561ce014beb4879f726fc48
Author: Peter Lisák <peter.lisak@firma.seznam.cz>
Date:   Thu Oct 22 10:19:49 2015 +0200

Change schedule priority of daemon/server in config
    
    The goal is to modify schedule priority and I/O scheduling class and
    priority of daemon/server via configuration.
    Setting is optional, default keeps current behaviour.
    
    Use case:
    Prioritize object-server to object-auditor, because all user's requests
    needed to be served in peak hours and audit could wait.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    DocImpact
    Change-Id: I1018a18f4706daabdb84574ffd9a58d831e68396

commit 44ba3c310a90f6db49417220ca979fd945ca1799
Author: Doron Chen <cdoron@il.ibm.com>
Date:   Wed Aug 10 09:29:44 2016 +0300

Added a test for delayed reap.
    
    The test schedules a 3-second delayed account reaping.
    The test checks that no reaping after an immediate reap, and that full reaping
    occurs 3 seconds later.
    
    Change-Id: I0ab954ed3c59d808f32d84dc53fd512fd0a651be

commit ddbab0509442e55ca6a68b9ef9a879e49e0e4f67
Author: Thiago da Silva <thiago@redhat.com>
Date:   Fri Aug 5 14:22:28 2016 -0400

add reminder how to run debug func tests
    
    added comments on how to run in_process and specific
    test cases
    
    Change-Id: I485755996b15753323d30de09914d35e262fcedc
    Signed-off-by: Thiago da Silva <thiago@redhat.com>

commit bd29a3e3c7e49b1d7749c605bc150de0af168fd4
Author: KATO Tomoyuki <kato.tomoyuki@jp.fujitsu.com>
Date:   Tue Aug 9 13:17:38 2016 +0900

Remove the duplicated word 'be'
    
    Change-Id: I3ff4e7135d8d10c62dfcde90f34befe328ac39b2

commit 8b5578c362125d0ec7efdc5742d185b344f892f0
Author: Petr Kovar <pkovar@redhat.com>
Date:   Mon Jul 25 18:34:21 2016 +0200

[install-guide] Include environment-networking
    
    environment-networking.rst and edit_hosts_file.txt were left out
    when swift-specific content was moved from OpenStack Installation
    Guide to swift install-guide.
    
    Change-Id: I334cca0634e3071a7ea285e6ad49ed9baaf7dca8
    Partially-Implements: blueprint projectspecificinstallguides
    Closes-Bug: 1605021

commit 4aa1ae61cb62d9f0e8ca6ceecabb0b9b40fedf96
Author: Or Ozeri <oro@il.ibm.com>
Date:   Wed Jun 15 19:56:03 2016 +0300

Raise 412 response on expirer
    
    Currently, the expirer daemon treats 412 (precondition failed)
    as successful DELETEs.
    
    On the other hand, it treats 404 as failed while reclaim_age
    (usually a week) has not passed.
    This patch unifies both cases to the same handling: waiting for
    reclaim_age to pass, then deleting the entry.
    
    The reason the expirer should not delete a 412 entry right away,
    is that it might be the case that 412 is returned because of
    a split brain, where the updated object servers are currently down.
    Same reason holds for a 404 response.
    
    Change-Id: Icabbdd72746a211b68f266a49231881f0f4ace94

commit c5ff9932a45cccadf89d9a99759a81eec2dd337f
Author: Shashirekha Gundur <shashirekha.j.gundur@intel.com>
Date:   Thu Aug 4 20:35:19 2016 +0000

NIT: fixing inconsistent naming of OpenStack Swift
    
    Throughout the manpages maintaining references to OpenStack Swift.
    
    Change-Id: I2a0c2658e10a92671bfc092c0a3abaddfd8cd7d9
    Closes-Bug: #1609687

commit a9fa5abdc37fe9ce40da9b6c310cef93dae1a781
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Fri Aug 5 14:28:22 2016 +0100

Tighten up direct client unit tests
    
    Make the tests for direct_get_container and direct_get_account
    verify all combinations of request parameters.
    
    Change-Id: I3b929ca83b37c32927b9bf619f445d698b9bdab9
    Related-Change: I846fc70ff3abdb1674152a8d9e0521c709f254c4

commit ce44ec45a4722681184e9a68f00b473abf723aa3
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Thu Aug 4 13:05:32 2016 +0800

Make log information format
    
    Change-Id: I4a7699ce17c136490327776c0a3c3be450526814

commit 3096dc6b1e299d2e0a372f7979158d1a4dd38144
Author: houweichao <houwch@gohighsec.com>
Date:   Thu Aug 4 19:31:18 2016 +0800

Make the logger information format
    
    The logger usually use the _() func to ensure
    the international message. This commit modify the
    logger format in proxy pkg.
    
    Change-Id: I9a8e118dc6ba55e462f326aebff280289e8ab4fe

commit fb5fcb189e32746f8c884cd53cb5239a384bc070
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Aug 2 21:50:45 2016 -0700

Fix encryption-delimiter interaction
    
    Previously, if a container listing produced `subdir` elements
    the decrypter would raise a KeyError.
    
    Additionally, update the functests so this sort of thing would
    have been caught at the gate.
    
    Closes-Bug: 1609904
    Change-Id: Idc1907d19f90af7a086f45f8faecee9fbc3c69c2

commit 4f94e71d554f885233da1a055bf7972353f3c6ec
Author: Paul Dardeau <pauldardeau@gmail.com>
Date:   Sun Jul 31 18:31:37 2016 -0400

add swift-oldies man page
    
    Closes-bug: #1607017
    Change-Id: I365edf77c5bf34e2e51d7f10c9ea1012c1bda8ed

commit 303e4a1d83bc94793a47756b9d0b3f169c1eeb23
Author: Michael Barton <mike@weirdlooking.com>
Date:   Thu Aug 4 03:05:40 2016 +0000

missing parens in functional test
    
    I was watching the logs and noticed it creates an object named:
    "<bound method type.create_ascii_name of <class 'test.functional.tests.Utils'>>"
    
    Change-Id: I8dcbb40125b84a914e3c01566ae9c3f08dc9ea0f

commit 425bb0d8b0f78feb362fac69cd75bfe8baa3f841
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Aug 3 16:46:48 2016 -0700

Fix stale docstring in SegmentedIterable.
    
    At some point, we added stuff to listing_iter, but didn't update the
    docstring. I noticed this while trying to write code using a
    SegmentedIterable when it wouldn't take my 3-tuples like the docstring
    claimed it would.
    
    Change-Id: I8f6667e97b1277f5b403a5f6fa7f9d708bb19249

commit 488f88e30abe2b9d67ce43697e3abc63c75699b7
Author: Gábor Antal <antal@inf.u-szeged.hu>
Date:   Fri Jul 15 14:35:54 2016 +0200

Use more specific asserts in test/unit/cli tests
    
    I changed asserts with more specific assert methods.
    e.g.: from assertTrue(sth == None) to assertIsNone(*) or
    assertTrue(isinstance(inst, type)) to assertIsInstace(inst, type)
    or assertTrue(not sth) to assertFalse(sth).
    
    The code gets more readable, and a better description will be shown on fail.
    
    Change-Id: I39305808ad2349dc11a42261b41dbb347ac0618a

commit 5cf24ecf165e82076db5c9e113b2127de7782a90
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Aug 2 10:57:59 2016 +0100

Fix link to docs from swift-dispersion manpages
    
    Change the link to the admin docs to point to #dispersion-report
    rather than non-existent #cluster-health.
    
    Change-Id: Ia2f4262c266201d4d555e7bedb8c5c2eb9fb1264

commit e0c7e6bf78c4bb0f2ad047a9d6f6926a2ae02d9a
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Aug 2 10:38:45 2016 +0100

Fix repeated OPTIONS in swift-dispersion-report manpage
    
    Remove the repeated OPTIONS section headers, which looks odd
    and is inconsistent with swift-dispersion-populate.
    
    Change-Id: I6d894e3b61002ddf7c0ea8a78cde226617eb11a6

commit d819ae00a5f589fc7cd11ddf95cfce7323d926e8
Author: Shashirekha Gundur <shashirekha.j.gundur@intel.com>
Date:   Mon Aug 1 22:13:58 2016 +0000

update swift-dispersion manpages to add policy-name
    
    This change adds -P / --policy-name option to the swift-dispersion
    manpages. Also tidied up a little removing some extra whitespace at the
    end of lines.
    
    Change-Id: Ic3372379994964e96258939580452f94fb575a06
    Closes-Bug: #1605686

commit e7f025f7fa56056ddad82aa5c601894b25c8927b
Author: maoshuai <fwsakura@163.com>
Date:   Mon Aug 1 21:39:50 2016 +0800

made link in README.rst more clear
    
    Escaping the underscore is not necessary in this case. See
    http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#inline-markup-recognition-rules
    
    Change-Id: I21d95d6baaf471246eb8a931c7df366634529512

commit 99a6f915ffc3aba2086e22dfba1f33b4fee46e81
Author: Nandini Tata <nandini.tata.15@gmail.com>
Date:   Fri May 20 17:41:29 2016 +0000

swift-ring-builder output corrected for ipv6
    
    Adjusted width of ip and port columns in swift-ring-builder command
    output to dynamically span to the longest ip or the longest port in
    the devices list. Also combined the port and ip address columns for
    better visual clarity. Took care of ipv6 format [ipv6]:port
    
    Modified the corresponding test case with expected output.
    
    Change-Id: I65837f8fed70be60b53d5a817a4ce529ad0f070e
    Closes-Bug: #1567105

commit 1533eb3f3fb358eea92a9f23c4558dd8a0577d1a
Author: Béla Vancsics <vancsics@inf.u-szeged.hu>
Date:   Fri Jul 15 12:22:30 2016 +0200

Reduce code duplication
    
    Reduced source code by extracting duplicated code
    (swift/cli/ringbuilder.py)
    http://openqa.sed.hu/dashboard/index?did=1&id=OpenStack%3Aswift,
    in 127~CloneClass
    
    Change-Id: Id1081363610075f306eff7cf003c3355f283f1d1
    Closes-Bug: 1536127

commit eb535904bac7c9f8a73b672471dd01f5971716a1
Author: gengchc2 <geng.changcai2@zte.com.cn>
Date:   Fri Jul 29 11:43:32 2016 +0800

modify the home-page info with the developer documentation
    
    update home-page info
    
    Change-Id: I625c25a8a5698d98174603c6fa2b42391471c03d

commit 0d41b2326009c470f41f365c508e473ebdacb11c
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Tue Jun 7 14:52:28 2016 +1000

Add end_marker and reverse options to direct_client
    
    Currently the direct_get_container and direct_get_account methods
    of the direct client don't support passing in the 'end_marker' and
    'reverse' params.
    
    This change adds support for these params in direct client.
    
    Change-Id: I846fc70ff3abdb1674152a8d9e0521c709f254c4

commit 66520146cfea2db77cbc4a1eaf5648a5122654ec
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Mon Jul 25 22:07:15 2016 -0500

Enable in-process func tests to optionally use encryption
    
    Running functional tests in the in-process mode uses
    the default value for the pipeline.  This patch adds support
    to specify the SWIFT_TEST_IN_PROCESS_CONF_LOADER variable
    to point to a labeled function that changes the proxy
    configuration for the functional test.
    
    The patch also adds a new tox environment
    func-in-process-encryption
    which runs with the environment variable
    SWIFT_TEST_IN_PROCESS_CONF_LOADER=encryption
    
    The motivation for this change is to put support in place for an
    upstream CI job that will functionally test using encryption
    middleware in the pipeline.  The gate job is proposed at:
    https://review.openstack.org/#/c/348292/
    
    Change-Id: I15c4b20f1d2be57ae21c69c614f6a9579145bee9

commit 5c9732ac8e797e9235ccd6df56253a802e517ab6
Author: Nandini Tata <nandini.tata.15@gmail.com>
Date:   Tue Jul 26 20:20:38 2016 +0000

Moved ipv4 & ipv6 validations to the common utils
    
    Validating ip addresses for ipv4 and ipv6 formats have more generic
    use cases outside of rings. swift-get-nodes and other utilities that
    need to handle ipv6 adrresses often require importing ip validation
    methods from swift/common/rings/utils (see Related-Change). Also,
    expand_ipv6 method already exists in swift/common/utils. Hence moving
    validation of ips also into swift/common/utils from
    swift/common/ring/utils.
    
    Related-Change: I6551d65241950c65e7160587cc414deb4a2122f5
    Change-Id: I720a9586469cf55acab74b4b005907ce106b3da4

commit aa2a84ba8a8d6ba08141492502b3872ad0d81c2d
Author: Rebecca Finn <rebeccax.finn@intel.com>
Date:   Wed Jul 13 19:09:39 2016 +0000

Check object metadata constraints after authorizing
    
    In the object proxy controller, the POST method checked the metadata of an
    object before calling swift.authorize. This could allow an auth middleware to
    set metadata that violates constraints. Instead, checking the metadata should
    take place after authorization.
    
    Change-Id: I5f05039498c406473952e78c6a40ec11e8b53f8e
    Closes-Bug: #1596944

commit 457cea864c4b49bcf79327369c0bc5ebc24668e2
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Fri Mar 11 16:15:17 2016 -0800

Handle IPv6 addresses in swift-get-nodes.
    
    The curl commands needed a little tweaking.
    
    Change-Id: I6551d65241950c65e7160587cc414deb4a2122f5
    Closes-Bug: 1555860

commit 8bf28c869d814c33f232d982420f713bbe629003
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Jul 28 11:09:48 2016 +0100

Fix broken link in associated projects doc
    
    Change-Id: I6f0cc1004a40e77345c641c0e5076f2f5dadb891

commit 6575ac0ea232f46bdea2a0ae1ad847b8d9062ac2
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Thu Jul 28 14:44:56 2016 +0800

Simple code and improve the readability of the code
    
    In a function,when a variable is defined at the front,it is unnecessary
    to define the variable again.
    
    Modify the variable name to improve the readability of the code.
    For example: container_nodes=container_info["nodes"] instead of
    containers=container_info["nodes"]
    
    Change-Id: I30b887fed08ff650076ce8ce33b1502bc1795b3f

commit 149a331f1cd2e2488bb0aab3c06c4c70cf74ece6
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Jul 27 09:08:11 2016 +0100

Additional test for container override etag preferences
    
    Adds one more scenario to the tests for preference of
    container etag override values in sysmeta or backend
    headers and footers.
    
    Change-Id: Iacdaec8c98c7001029163a9d50321a13dc8d5a19
    Related-Change: I074fbecb6440fb1d04279cd892d38d2acc44b47d

commit fbf0e499171d0880d0dc745767ac3a531d08c0c6
Author: Victor Stinner <vstinner@redhat.com>
Date:   Tue Jul 26 19:42:46 2016 +0200

monkey_patch_mimetools() now does nothing on py3
    
    The mimetools module has been removed from Python 3: modify
    monkey_patch_mimetools() to do nothing on Python 3.
    
    Skip test_monkey_patch_mimetools() on Python 3.
    
    Change-Id: I50f01ec159efedbb4df759ddd1e13928ac28fba6

commit 7cc2392611cccb48100cafc1b4ec5fdf03855b3d
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Jul 26 18:12:26 2016 +0100

Document how to run a single functional test
    
    Change-Id: Icabc5a8316f5e8fd887bb42358ad03e9c43d0765

commit cf1f7af38751f280906954b7a01ee4f273336264
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jul 25 12:30:34 2016 -0700

Use self.ts more consistently in obj.test_server
    
    A make_timestamp_iter was added into setUp in the Related-Change.
    
    There was a couple of different tests that were using timestamp iters
    in different ways, consistently we apply the use of next(self.ts)
    which I believe is becoming more common/standardized in unitests.
    
    Related-Change: I074fbecb6440fb1d04279cd892d38d2acc44b47d
    Change-Id: Ib6b883afec242355ae08c50c1e685a20e5efadc7

commit a97537e158a78b7c49f11aef2756fa65093a05dd
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jul 25 12:22:20 2016 -0700

Split up backend/sysmeta header/footer preference tests
    
    The change in preference to the Related-Change is simply to prefer
    sysmeta overrides in the headers to backend overrides in the footers:
    
    Before:
    sysmeta footers > backend footers > sysmeta headers > backend headers
    
    After:
    sysmeta footers > sysmeta headers > backend footers > backend headers
    
    This change just breaks up the tests to try to make it more obvious
    what already worked and what has changed.  The justification seems to
    be that overrides in sysmeta headers only work on policies that don't
    send backend footers, but sysmeta overrides should always have a
    higher preference than backend overrides.
    
    Related-Change: Idb40361ac72da51e1390dff690723dbc2c653a13
    Change-Id: I074fbecb6440fb1d04279cd892d38d2acc44b47d

commit 77e476376c954d041dfb3f0cca8c0557f8482964
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Jul 26 17:44:56 2016 +0100

Mention SWIFT_TEST_DEBUG_LOGS in development guide
    
    Change-Id: If764de0a28f5afb858b3e892b35fe5fa147a0650

commit 7958638e8f7800813fe8ec5bb860c14e3b81c6c0
Author: Ellen Leahy <ellen.mar.leahy@hpe.com>
Date:   Thu Jul 21 11:27:59 2016 +0100

Added quotes to example echo in swift-temp-url
    
    If the curl command is used exactly as in the help, the ampersand
    in the signature is interpreted as an operator and the curl
    command breaks. I am aware of developers who have wasted a lot of
    time because of this.
    
    Change-Id: I6468c9a098b56db8242a2cf2c23b7a4857bd8574

commit a81d60472fc237234bcd4798d9f37d554ac0ba3f
Author: Victor Stinner <vstinner@redhat.com>
Date:   Tue Jul 26 12:36:50 2016 +0200

Fix Python 3 issues in diskfile
    
    * Fix bytes vs Unicode issues
    * On Python 3, encode JSON to UTF-8 and decode it from UTF-8
    * Open files in binary mode to avoid Unicode issues
    * test_auditor: use bytes for content, open files in binary mode
    
    Change-Id: Ifa84001493cfb57975d3b140b0d7e09020504bca

commit 699953508ad1fd82c221e57bccfb1de8bf7a7e31
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Jun 9 06:17:22 2016 +0000

Add doc entry to check partition count
    
    An high or increasing partition count due to storing handoffs can have
    some severe side-effects, and replication might never be able to catch
    up. This patch adds a note to the admin_guide how to check this.
    
    Change-Id: Ib4e161d68f1a82236dbf5fac13ef9a13ac4bbf18

commit c1c18da82c8e8d03c4e5b52910661632f8acfe46
Author: cheng <shcli@cn.ibm.com>
Date:   Mon Jul 25 13:23:04 2016 +0000

check _last_part_moves when pretend_min_part_hours_passed
    
    pretend_min_part_hours_passed do things like this:
    self._last_part_moves[part] = 0xff
    
    this will throw exception if self._last_part_moves is None.
    
    this patch is to check self._last_part_moves to prevent exception.
    Closes-bug: #1578835
    
    Change-Id: Ic83c7a338b45bfcf61f5ab6100e6db335c3fa81a

commit bf17d968008addadfb06fe35e83ac9de38ba2af4
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Jul 25 15:02:03 2016 -0700

Use extract_swift_bytes in override_bytes_from_content_type
    
    About half the logic was the same, and it seems better to just implement
    that once.
    
    Change-Id: I350da34ef7a3cd0cb74f585f4691992ae64c7eab

commit 5677a04c8f5d46da80ac4cf8be135549b422772c
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jun 23 13:53:49 2016 +0200

Python 3: Fix usage of dict methods
    
    * Replace "c = dict(a.items() + b.items())" with
      "c = dict(a); c.update(b)". It works on Python 2 and Python 3, and
      it may be a little bit more efficient on Python 2 (no need to
      create a temporary list of items).
    * Replace "dict.values() + dict.values()" with
      "list(dict.values()) + list(dict.values())": on Python 3,
      dict.values() is a view which doesn't support a+b operator.
    
    Change-Id: Id5a65628fe2fb7a02c713b758fcaa81154db28a0

commit e6776306b7d486ebd35c8f388b0ff6db51b0752b
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jun 23 13:42:01 2016 +0200

Python 3: fix usage of reload()
    
    Replace reload() builtin function with six.moves.reload_module() to
    make the code compatible with Python 2 and Python 3.
    
    Change-Id: I7572d613fef700b392d412501facc3bd5ee72a66

commit 1eb96397e7a6f477ba31df85eda892769a8a182e
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Fri Jan 15 03:53:01 2016 -0800

Fix EC ring validation at ring reload
    
    Swift EC has a strong constraint about the ring must have a number of
    replicas which fits ec_k + ec_m. That is validated when servers waking
    up. However, Swift has more chance to load such an invalid ring when
    a request comming, calling some node iteration like get_nodes,
    get_part_nodes or so, and no ring validation is there.
    
    This patch moves ring validation from policy validate_ring into the ring
    instance as validation_hook that will run at ring reload. Since this patch,
    ring instance will allow to use the old ring if the reload is not fourced.
    
    Note that the exception if invalid ring found was changed from
    RingValidationError to RingLoadError because RingValidationError is a
    child of RingBuilderError but the ring reload is obviously outside of
    "builder".
    
    Closes-Bug: #1534572
    
    Change-Id: I6428fbfb04e0c79679b917d5e57bd2a34f2a0875

commit 3a1a198780433e6ebf2bcc3862bd63c7d033930d
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Sun Jul 3 22:03:20 2016 -0700

Raise ValueError if empty value coming into encrypt_header_val
    
    encrypt_header_val is used to translate a raw header value into an
    encrypted value. Semantically, a header with an empty value won't be stored
    and all callers seem to remove such a header before calling
    encrypted_header_val.
    
    So if no reason for returning ('', None), I think it's better to change it to
    raise ValueError not to cause another error for users of the return value.
    (e.g. dump_crypto_meta)
    
    Plus this patch addes a few unit tests for those cases above.
    
    Change-Id: Ic1237f4afb8c0e466be5ce59fe31b667c39242b0

commit 2876f59d4cdbafc297d79f4e6295f05e3448ae47
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Jul 20 18:16:27 2016 -0700

Cache fragment size for EC policy
    
    ECStoragePolicy.fragment_size is never changed on running Swift because
    it is from ec_segment_size and ec_type defined in swift.conf statically
    so let's cache the value after retrieving the value from the pyeclib driver.
    
    And more, pyeclib <= 1.2.1 (current newest) has a bug [1] to leak the reference
    count of the items in the returned dict (i.e. causes memory leak) so that
    this caching will be mitigation of the memory leak because this saves the call
    count fewer than current as possible.
    
    Note that the complete fix for the memory leak for pyeclib is proposed at
    https://review.openstack.org/#/c/344066/
    
    1: https://bugs.launchpad.net/pyeclib/+bug/1604335
    
    Related-Bug: #1604335
    Change-Id: I6bbaa4063dc462383c949764b6567b2bee233689

commit afe3968dd158d829c667764a3acd223b52aa5756
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Jul 6 11:27:58 2016 +0100

Copy headers correctly when copying object
    
    Fix copy middleware so that all client-defined object
    headers that object servers allow to be persisted are copied.
    For example, content-encoding and content-disposition will
    now be copied.
    
    Fix treatment of x-fresh-metadata header so that, when it is
    used, new object sysmeta is applied to the object copy in the
    same way as a copy without x-fresh-metadata.
    
    Remove unnecessary passing of original request headers to
    sink PUT request constructor: passing the environ is sufficient
    to have the new request inherit the original's headers.
    
    Add tests for this change and to verify that content-type
    gets either copied or updated if supplied with the copy
    request.
    
    Add tests for x-fresh-metadata treatment.
    
    Closes-Bug: #1391826
    Closes-Bug: #1600247
    Co-Authored-By: Thiago da Silva <thiago@redhat.com>
    Change-Id: I917fb0b4e831c13e04ade1c5e0b9821802dec967

commit 26d91f2b1006a3affff963d6aa8d0d0ced134723
Author: Lokesh S <lokesh.s@hp.com>
Date:   Wed Jul 20 13:49:29 2016 +0000

Python3 eventlet patched httplib _MAXHEADERS
    
    This change patches the correct eventlet monkey patched httplib's
    _MAXHEADERS as required by Swift.
    
    It also makes the configparser inside the copy middleware py3
    compatable.
    
    Change-Id: I2f7dbcecbbecd7cb69b7031faa39f35bcfadcfc8

commit 873331185713b2e2466456e8cfcdb02ec37ff955
Author: John Dickinson <me@not.mn>
Date:   Wed Jul 20 11:09:35 2016 -0700

Prevent CPU spinning when there are no children
    
    If you deploy an object server but have no rings at all (and are using
    servers-per-port), then the CPU will spin as it checks for child
    processes since there are actually no child processes to check.
    
    This patch adds a sleep so that the CPU doesn't spin.
    
    Change-Id: Iece62367aa2481a21752144b1f4477a3713282fe

commit 49f250736d0d9fa0a0ff166caffcacc58d24fc1d
Author: Lokesh S <lokesh.s@hp.com>
Date:   Wed Jul 20 19:00:00 2016 +0000

Python3 fixes generator object issue
    
    Fixes generator' object has no attribute
    'next' issues
    
    Change-Id: I1f21eaed0ae7062073438503d3f6860d8b4f36c8

commit cdf505a50c107c14e626f308bd26eee9b4b9f305
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jul 20 17:59:25 2016 +0000

Make swift-oldies py3-compatible
    
    Change-Id: I0388f4738966bc453e922e9598ff9df60ecda4eb

commit d9b765320d58571b6b123839f8555f95c20a6bbf
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Tue Jul 19 21:37:54 2016 +0000

Fixed Typo in updater.py
    
    Change-Id: Ic0b5445e313924c683e9889d94569c2554fd0b8b

commit f33742127339a39c1b7bd9bf26f31c2cd9fbb341
Author: Cheng Li <shcli@cn.ibm.com>
Date:   Tue Jul 19 17:25:22 2016 +0800

Change assertTrue to assertEqual
    
    In test_ringbuilder.py, there is one assertTrue should be
    replaced with assertEqual.
    
    Change-Id: I9a0e4a7363a5e16cc9b6df045953dfbb4f9dbd07
    Closes-bug: #1604320

commit aaa631558785e447c9655b9ada7f894035999425
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Tue Jul 19 08:27:04 2016 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Ica92e45dbc3f83ec806e092fd3ea5ada636ab7e9

commit cc0016399d78426dd4819fb97a508849194a7f4e
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jul 18 14:01:57 2016 -0700

Remove red herring from logs
    
    The object replicator can log some junk about the cluster ip instead
    of the replication ip in some specific error log lines that can make
    you think either you're crazy or your rings are crazy.
    
    ... in this case it was just the logging was crazy - so fix that.
    
    Change-Id: Ie5cbb2d1b30feb2529c17fc3d72af7df1aa3ffdd

commit c7e5afb9c375b260c65782ce72b10a7739ad3bbb
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Mon Jul 18 19:49:44 2016 +0000

Fixed typo in reaper.py
    
    Change-Id: I9b98da30e5f934164e490beb4d6cde840f08832a

commit bc09be4375b45cd98fd14f91f25b71bebc8bd08c
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Mon Jul 18 15:01:21 2016 +0800

Make comparision simplely
    
    For example: a>b and a<=c is equal to b<a<=c
    
    Change-Id: Iae1532f0946c6d4aa7321f3957820b486869c59f

commit ecce7947dda5b72757f88610ca83f2c0e78cf82a
Author: zheng yin <yin.zheng@easystack.cn>
Date:   Sun Jul 17 18:02:05 2016 +0800

Add log format to i18n
    
    Change-Id: I3aff0de418b52b1f16f5863c95ad2700678143ae

commit 45bde710a92823cb96b79755a86fe6076cb06b32
Author: Maria Malyarova <savoreux69@gmail.com>
Date:   Fri Jul 15 21:38:14 2016 +0300

Simplify chained comparison
    
    For example: a < b and b <= c is equal to a < b <= c
    
    Change-Id: I91ceb194bce60f6160ebdf0aadf0e8f0d7a35975

commit e278179b082336fb0870ebccecf16c5d02e5ae0c
Author: Gábor Antal <antal@inf.u-szeged.hu>
Date:   Fri Jul 15 14:02:38 2016 +0200

Use more specific asserts in functional tests
    
    I changed asserts with more specific assert methods.
    e.g.: from assertTrue(sth == None) to assertIsNone(*) or
    assertTrue(isinstance(inst, type)) to assertIsInstace(inst, type) or
    assertTrue(not sth) to assertFalse(sth).
    
    The code gets more readable, and a better description will be shown on fail.
    
    Change-Id: I80ec96e0b729bef38213a6be4ff4b6eb65c7612d

commit 75a58a6dd8ef439c613226b562eb3ab766d16c5e
Author: Gábor Antal <antal@inf.u-szeged.hu>
Date:   Fri Jul 15 15:02:00 2016 +0200

Use more specific asserts in test/unit/proxy tests
    
    I changed asserts with more specific assert methods.
    e.g.: from assertTrue(sth == None) to assertIsNone(*) or
    assertTrue(isinstance(inst, type)) to assertIsInstace(inst, type)
    or assertTrue(not sth) to assertFalse(sth).
    
    The code gets more readable, and a better description will be shown on fail.
    
    Change-Id: If6aad8681aab7c9a41d65a4f449d8abbe3e64616

commit 77d6d015f694eb4fdafc3b1ae6ff1caa9d0cbdb1
Author: Gábor Antal <antal@inf.u-szeged.hu>
Date:   Fri Jul 15 14:46:41 2016 +0200

Use more specific asserts in test/unit/common/ring
    
    I changed asserts with more specific assert methods.
    e.g.: from assertTrue(sth == None) to assertIsNone(*) or
    assertTrue(isinstance(inst, type)) to assertIsInstace(inst, type)
    or assertTrue(not sth) to assertFalse(sth).
    
    The code gets more readable, and a better description will be shown on fail.
    
    Change-Id: I9531c9939aa7c2dac127b5dc865b8d396dab318f

commit 5d02b9578e48a2eacc5111a73a399642eb28fe15
Author: Béla Vancsics <vancsics@inf.u-szeged.hu>
Date:   Fri Jul 15 12:40:29 2016 +0200

Reduce code duplication
    
    Reduced source code by extracting duplicated code
    (swift/cli/ringbuilder.py)
    
    Change-Id: Ibd000df1dc9042e31b65b000199dff4a645e63b4

commit 88238108f824e14fca8ca6adb3ec0e72df87fdf9
Author: John Dickinson <me@not.mn>
Date:   Sun Jul 10 10:00:09 2016 -0700

authors and changelog updates for 2.9.0 release
    
    Change-Id: I3c3e779227aad1df6abb517817355c6732e4a2af

commit 90627f903adaecede63b90a3be2270178b3685e3
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Mon May 23 21:10:08 2016 +0000

Add region in ring structure & deployment guide
    
    Deployment guide does not talk about the region. Also, it does not
    specify that regions and zones need to be ints.
    
    This patch adds brief description about region and changes numbers
    to int. Also, adds region in the document that talks about ring data
    struture.
    
    Change-Id: I04ce42fb3e5c1f08e7f7ff6be23482cee8bdeb71
    Partial-Bug: #1583551

commit 6740a7badd795d2ccc477472581b56b628b96818
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jul 1 14:42:43 2016 -0700

Add keymaster_config_path option to keymaster
    
    Also, tighten up the format checks on root secrets.
    
    Change-Id: I1cd9a97c4e8d87d7c065866e7ad3a9e748ff19ab

commit 5cd57dc3572bccd443d74d0e18ea64a221073dac
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Tue Jul 14 15:13:06 2015 +0900

Improve Keystone v3 token support
    
    Now keystoneauth uses HTTP_X_TENANT_NAME/ID only even if v3 token uses for
    backward compatibility. There is no problem with current behavior because
    keystonemiddleware set same value on the headers but the headers are
    specified as deprecated so this patch allows to support HTTP_X_PROJECT_NAME/ID
    in addition to HTTP_X_TENANT_NAME/ID.
    
    Change-Id: Ie5e02067a59e18f1ac215f51429863bdd42f729f

commit 5817f00005d2c1761499c3d6e5e0d428ef238c77
Author: Cheng Li <shcli@cn.ibm.com>
Date:   Sun Jul 10 20:33:05 2016 +0800

make 0 be avaiable value of options
    
    value 0 is regard as not available by swift-ring-builder
    $ swift-ring-builder testring add --region 0 --zone 1
    --ip 127.0.0.2 --port 6000 --device sdb --weight 100
    
    Required argument -r/--region not specified.
    The on-disk ring builder is unchanged.
    
    this patch is to make value 0 available.
    
    Change-Id: Id941d44d8dbfe438bf921ed905908b838c88a644
    Closes-bug: #1547137

commit 6b6fa693499827b6bddf5a17d0787315ab3c5011
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Sun May 15 14:36:57 2016 +0000

Add description of server options (all, main, rest) for swift-init help
    
    Running swift-init with -h, --help, or no arguments
    displays help for the command. The help does not
    document the 'main', 'all', and 'rest' options.
    These are documented in the man page.
    
    This patch adds all these server options in the
    help of swift-init.
    
    Change-Id: I8e27589912ae72ace14c955e66b86942bc23d9f7
    Closes-Bug: #1580722

commit 9890184ea9378fbba1cb76e861f5a20bdd36b7c9
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jul 7 18:00:05 2016 +0000

Turn on H233 and start using print function
    
    As much as anything, I'm just tired of seeing a bunch or piecemeal
    fixes.
    
    Note that we *need* to include
    
       from __future__ import print_function
    
    in order to support things like
    
       print()  # Would print "()" (the repr of an empty tuple) otherwise
       print(foo, end='')  # Would SyntaxError
       print(bar, file=sys.stderr)  # Would SyntaxError
    
    Change-Id: I8fdf0740e292eb1ee785512d02e8c552781dcae1

commit 54ed0842344b6b06d5d874a297754fefcd993585
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Fri May 20 16:26:15 2016 +0000

Add region in swift-ring-builder add
    
    In the swift deployment guide, region is missing from the syntax of
    adding a new device to the swift-ring-builder.
    
    This patch adds region in the syntax.
    
    Change-Id: I43e247c92d461efd530c0f82ca3daddcb9e2ba5b
    Closes-Bug: #1584127

commit ffaef489c6d1420ba6f4baf50b8fe84aa3d86319
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Fri Jul 1 14:39:35 2016 +0100

Add encrypter and decrypter links to middleware.rst
    
    Drive-by fix for crypto filter_factory test.
    
    Add note to encryption doc to highlight that root secret
    should not be changed (follow up on earlier review comment).
    
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    
    Change-Id: I9776cddd4d045408325342983e285a00c992bfae

commit c21375227165077d48bb7ef4e64a11f46d68b696
Author: Victor Stinner <vstinner@redhat.com>
Date:   Wed Jun 29 09:30:34 2016 +0200

Update dnspython to 1.14
    
    dnspython 1.14 adds Python 3 support and so can now be used on Python
    2 and Python 3. Drop dnspython3 dependency.
    
    Change-Id: I0a860b03800aeeed4375f528e6bf9cca57129db7

commit dcee7028019b89703fb5e25dd59fdb6fd882129a
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jun 23 13:40:42 2016 +0200

Python 3: fix urllib import
    
    Replace urllib.quote import with six.moves.urllib.parse.quote, so the
    code works on Python 2 and Python 3.
    
    Change-Id: I17e9cd9668661b6a67f33db83e0cbfc8ea6e3ca6

commit da317f01c6aeece5c4c4bccb49972a84a3b5580f
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jul 7 17:58:58 2016 +0000

Run flake8 against scripts in bin
    
    Just having a `flake8 --filename=swift* bin` command is insufficient.
    
    Change-Id: Ia2a5c364e52d9972d31e6b5e22366503894b720d

commit 6f230c7ea0ffd3b0b85c4bf5701e57e0f9fb1570
Author: Nandini Tata <nandini.tata.15@gmail.com>
Date:   Thu Jul 7 21:24:52 2016 +0000

Fixed inconsistent naming conventions
    
    Fixed naming conventions of Keystone, Swift and proxy servers in
    the docs.
    
    Change-Id: I294afd8d7bffa8c1fc299f5812effacb9ad08910

commit b6be925cf59e0cf719307cbcdda876c8d135eda5
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 23 12:22:02 2016 -0700

Tighten memcached tests
    
    Check flags set and key used; stop relying on dict.values()
    
    Change-Id: Ibf9228dabd66ae98fb3b64050ccd46f5032d0df9

commit bfc8c59a08954f57d8f64ca66db9a1ba72207710
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Jul 7 18:35:48 2016 +0100

Add encryption package requirements to SAIO instructions
    
    libssl-dev/openssl-devel are already listed in other-requirements.txt;
    add them to installation instructions in the SAIO docs.
    
    Change-Id: I3dc07213ff8dac1299d3eb68d3448a77e15c79af

commit 008a037a3611734c2ba43c9aa0fe413bf5a2fa42
Author: Maria Malyarova <savoreux69@gmail.com>
Date:   Thu Jul 7 20:13:03 2016 +0300

Another amendment with missing parenthesis
    
    TrivialFix
    
    Change-Id: Id44ae27bc39ea97be9eb092f8a99a06056b86392

commit a53b12a62c0da2a654d13a26f75c7dc214c13e6a
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Wed Jun 8 15:33:22 2016 +0100

Don't encrypt update override etags for empty object
    
    Fix an anomaly where object metadata for an empty object has no
    encrypted etag, but if the encrypter received a container update
    override etag in footers or headers then it would encrypt that,
    so we'd have encrypted metadata in the container listing but not
    in the object metadata. (Empty object etags are not encrypted
    because the object content is revealed by its size anyway).
    
    This patch changes the override handling to not encrypt override
    etags that correspond to an empty object, with one exception: if
    for some reason the received override etag value is that of an empty
    string but there *was* an object body, then we'll encrypt the
    override etag, because it's value is not obvious from the object
    size.
    
    Change-Id: I8d7da34d6d98f351f59174883bc4d5ed0416c101

commit 9e82891f08da4fc933deef7839b9c205b7fc51b7
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Thu Jul 7 15:21:01 2016 +0000

Fix typo in object-server.conf and container-server.conf manpage
    
    Change-Id: Iffad70b2fd901b305dc66d363039b7df44d619da

commit 4c0a1481f147b51cf67bdaaa9fc0101ad1b63620
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Thu Jul 7 14:56:10 2016 +0000

Fix typo in the account-server.conf manpage
    
    Change-Id: I4e7bb85ce746fcb1ec3a4cbf534761e4e47634c9
    Closes-Bug: #1599888

commit ca2f6d13b6aa79d5e3c184955280b4a3ea17cd7d
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Thu Jul 7 11:31:31 2016 +0100

Fix unicode errors in object controller logging
    
    Change swift.proxy.server.Application.error_occurred()
    to decode message as utf-8 in same way that the
    exception_occurred() method was changed in [1].
    
    This prevents a unicode error when logging error responses
    in swift.proxy.controllers.base.Controller._make_request()
    for paths that have non-ascii characters. Although the unicode
    error is currently caught by a surrounding except clause, the
    logging and error limiting treatment is different for ascii
    vs non-ascii paths. This patch makes them consistent.
    
    Fix the server type reported in _make_request() to be
    the correct server type, not always 'Container Server'.
    
    Fix path arg passed to _get_conn_response in
    swift.proxy.controllers.obj.BaseObjectController to be req.path
    rather than req.
    
    Add unit tests for error_occurred() being called with non-ascii
    paths and extend tests for exception_occurred() (see Related-Bug).
    
    [1] Change-Id: Icb7284eb5abc9869c1620ee6366817112d8e5587
    
    Related-Bug: #1597210
    Change-Id: I285499d164bff94835bdddb25d2af6d73114c281

commit e5a6d458829b367ecc16af54a44f629cb1fdcd68
Author: Christian Schwede <cschwede@redhat.com>
Date:   Tue Jun 7 10:35:18 2016 +0000

Add ringbuilder tests for --yes option
    
    Also added a Timeout class to test.unit to wrap possible long-running
    functions. For example, if there is some regression and the "--yes"
    argument is no longer evaluated correctly and the test excepts some
    keyboard input, it will be terminated after a few seconds to ensure
    there is no long-running blocker on the gate.
    
    Change-Id: I07b17d21d5af7fcc594ce5319ae2b6f7f58df4bb

commit 2be1d6a77e9fcd4c92d487ba7116407ec5389723
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Thu Jul 7 21:28:07 2016 +0900

Remove an unused variable from tests.py
    
    The varialbe 'size' in TestFile.testMetadataNumberLimit is not used.
    This patch remove the variable from the test.
    
    Change-Id: I255a1dcee12bb6b8dec6ff26ed7edf93ab2acf64

commit b53214e30b1fb19336b77a19982c84c2dc5dce7f
Author: yuyafei <yu.yafei@zte.com.cn>
Date:   Thu Jul 7 18:27:21 2016 +0800

Correct reraising of exception
    
    When an exception was caught and rethrown, it should call 'raise'
    without any arguments because it shows the place where an exception
    occured initially instead of place where the exception re-raised.
    
    Change-Id: I326dd8eaf221cbf3729beedaff81b416c59ae2e6

commit cf8b93918cc8a46ebd9035028847d8009d362794
Author: Petr Kovar <pkovar@redhat.com>
Date:   Wed Jun 15 18:35:05 2016 +0200

Add install-guide for swift
    
    This adds swift-specific contents from the OpenStack Installation Guide
    in the swift repo per [1]. A separate change will remove the swift contents
    from the OpenStack Installation Guide for Newton per [2].
    
    The swift install-guide structure is based on the Install Guide
    Cookiecutter [3].
    
    Also adds tox.ini environment for install-guide and adds
    openstackdocs-theme to test-requirements.txt.
    
    [1] http://specs.openstack.org/openstack/docs-specs/specs/newton/project-specific-installguides.html
    [2] http://specs.openstack.org/openstack/docs-specs/specs/newton/installguide.html
    [3] http://git.openstack.org/cgit/openstack/installguide-cookiecutter/
    
    Change-Id: I59b92eebaf5acc657b97bcf10d9ff2cf2db05885
    Partially-Implements: blueprint projectspecificinstallguides
    Depends-On: Ifebc65b188c4f2ba35b61c0deae5ec24401df7f9

commit de51a6db36bcd965a7fa7505380100497ae3c66a
Author: yuyafei <yu.yafei@zte.com.cn>
Date:   Tue Jul 5 15:04:24 2016 +0800

Add __ne__ built-in function
    
    In Python 3 __ne__ by default delegates to __eq__ and inverts the
    result, but in Python 2 they urge you to define __ne__ when you
    define __eq__ for it to work properly [1].There are no implied
    relationships among the comparison operators. The truth of x==y
    does not imply that x!=y is false. Accordingly, when defining
    __eq__(), one should also define __ne__() so that the operators
    will behave as expected.
    [1]https://docs.python.org/2/reference/datamodel.html#object.__ne__
    
    Also remove class SubStringMatcher becasue this class isn't used
    following commit 7035639dfd239b52d4ed46aae50f78d16ec8cbfe.
    
    Change-Id: Ia2131f72a79226b0c2f3662b84661eb870d1d692

commit bcd9a58d3c30ce554bfbe7fee5bc851e9feccaa0
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jul 6 14:50:56 2016 -0700

Fix X-*-Container-Update-Override-* header/footer precedence
    
    Previously, all footer overrides (whether from the X-Backend-* or
    X-Object-Sysmeta-* namespace) would take priority over any header
    override.
    
    However, middleware should be able to set a Sysmeta override without
    needing to worry about whether it's working with a replicated policy
    (where setting it in headers will suffice) or an EC policy (where it
    would need to install a footers callback). This could be mitigated by
    *always* installing a footer callback, but doing so would incur
    additional overhead that would otherwise be unnecessary.
    
    Change-Id: Idb40361ac72da51e1390dff690723dbc2c653a13

commit a2afabf283811cdfd66b98add0df626008682ff8
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Jul 1 09:54:32 2016 -0700

Add basic functests for user object metadata
    
    Change-Id: I3c3b7d051a48449400e47e366461674bed9318c5

commit aa12901edadb5da21d96e36cee2e8f4f7b8273be
Author: Maria Malyarova <savoreux69@gmail.com>
Date:   Tue Jul 5 17:14:45 2016 +0300

Added missing parenthesis in print calls
    
    Upd. Import print_function from __future__
    TrivialFix
    
    Change-Id: Ibcda2c7e4ddbdff2420502dfd7d17db01f3c8056

commit 3781843cb3b32598cf66a5455d9c61c259fe77fa
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Jul 1 16:13:05 2016 -0700

Fix gettext_ calls
    
    Change-Id: I80e7d204f78620c6eaf63bfad18588c4096529b8

commit 7568ea5dd9f5a728f482076eeb8e612c31c88ce6
Author: Brian Cline <bcline@us.ibm.com>
Date:   Wed Jun 29 03:32:09 2016 -0500

Prevent down nodes failing PUTs with non-ascii obj names
    
    On an object PUT with a non-ascii name, if we hit some kind of
    exception speaking to only one object-server of the N we try to
    connect to, we try to log it -- but this causes an exception when
    interpolating the UTF-8 encoded path iff the message template is
    unicode.
    
    Since this is essentially an exception within an exception handler,
    this fails the entire request with a 500 error -- even though the
    other nodes may have been just fine. This occurs before it attempts
    a handoff node.
    
    The simplest way to reproduce this is by running func tests against
    a small cluster where one of the object nodes is not running
    
    N.B. The locale of the node does not matter because the message
    template is interpolated with node/device data from the Ring which is
    always unicode because of json.
    
    This includes an update to the FakeRing used by unittest
    infrastructure to ensure that the FakeRing devices make a round-trip
    through json to ensure consistent typing with real Rings.
    
    Change-Id: Icb7284eb5abc9869c1620ee6366817112d8e5587
    Closes-bug: #1597210

commit 401311ff6a2ab93aca772e6be027e7098be5a906
Author: Tim Burke <tim.burke@gmail.com>
Date:   Tue Jul 5 10:24:02 2016 -0700

Have py35 tox env match py34
    
    Very few of our tests can actually be run under py3. The ones that can
    should still pass on py35, though.
    
    Change-Id: Iaf9aaa296e3b21aa0ee513c479a50f3796787f32

commit d60662b2eb46cdd5d5ec591435d11beafb66ca01
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 30 14:32:35 2016 -0700

Only use Timeout if we don't have a final_resp
    
    I'm sure the Timeout context manager is relatively cheap, but it can't
    be free.
    
    Change-Id: I71c0c5944ec372e9b983021dd024de0c5aa1ded2

commit f9d5a8683d0b5204157c5d97aa039056612d4347
Author: yuyafei <yu.yafei@zte.com.cn>
Date:   Tue Jul 5 16:42:00 2016 +0800

Remove white space between print and ()
    
    TrivialFix
    
    Change-Id: I0dca3493d43ee8642ae6d2f55597013eef261026

commit c84a3c4d967a2951da50b395445e4a282b9debc4
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 30 15:28:24 2016 -0700

Stop digging for publicly_accessible ourselves
    
    Also, make request method calling in obj.server consistent with change
    3944d8 to account & container
    
    Change-Id: I893d77a06793a5eeafac203a45971e96425afb96
    Related-Change: I2f7586f96b41a97e6ae254efc83218b3b5c6cc9e

commit f36bc513c5e0029b90207d7a2dec81965eed8300
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Jun 7 15:08:54 2016 +0100

Add encryption overview doc
    
    Include a note in container-sync docs pointing to specific
    configuration needed to be compatible with encryption.
    
    Also remove the sample encryption root secret from
    proxy-server.conf-sample and in-process test setup. Remove encryption
    middleware from the default proxy pipeline.
    
    Change-Id: Ibceac485813f3ac819a53e644995749735592a55

commit 96a0e077532c3227b9290af7d74a0b42ee08e8de
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Tue Jun 7 15:01:32 2016 +0100

Enable object body and metadata encryption
    
    Adds encryption middlewares.
    
    All object servers and proxy servers should be upgraded before
    introducing encryption middleware.
    
    Encryption middleware should be first introduced with the
    encryption middleware disable_encryption option set to True.
    Once all proxies have encryption middleware installed this
    option may be set to False (the default).
    
    Increases constraints.py:MAX_HEADER_COUNT by 4 to allow for
    headers generated by encryption-related middleware.
    
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    Co-Authored-By: Christian Cachin <cca@zurich.ibm.com>
    Co-Authored-By: Mahati Chamarthy <mahati.chamarthy@gmail.com>
    Co-Authored-By: Peter Chng <pchng@ca.ibm.com>
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    Co-Authored-By: Jonathan Hinson <jlhinson@us.ibm.com>
    Co-Authored-By: Hamdi Roumani <roumani@ca.ibm.com>
    
    UpgradeImpact
    
    Change-Id: Ie6db22697ceb1021baaa6bddcf8e41ae3acb5376

commit c9b4d54972587f5f72c8c15820a85433b279a0c5
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 30 14:36:39 2016 -0700

Change elifs to ifs
    
    John seemed to have some misgivings about using elif.
    
    Change-Id: I39962607cf2a8f90353020f67979e92b48959dd6
    Related-Change: I7732f13b06c8826537b8f8230a2785607790b8e1

commit 6970099a76a8713da7744da40b462985b2c82a6a
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jun 29 14:30:39 2016 -0700

Fix intermittent bulk delete unit test failures
    
    Change-Id: I0822b14d7b1ddae5fe0cc567c7cbaf544cb081ee
    Closes-Bug: 1588414

commit 57ac316b0324a50f2313a125e8ae6c582953a1bd
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jun 29 13:21:31 2016 -0700

Fix intermittent SLO unit test failures
    
    Now that we have concurrent deletes, the order is not guaranteed.
    
    Change-Id: Ib833306a07ee0755a80501d5294eaa87b2347dc0

commit 365171395ea10def59d40e7fcc4d0f18c51bbe7b
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Jun 29 12:22:22 2016 -0700

Remove some unnecessary error handling in healthcheck
    
    ...as well as an unused class variable.
    
    Change-Id: If1091f420b0bcf34c37e49b13f59b229e8deecc6

commit 4a9f7378ec4d73c8bf16748e24bec45aa503b08e
Author: zhengyao1 <zheng.yao1@zte.com.cn>
Date:   Fri Jun 24 17:34:26 2016 +0800

make print python3 compatible
    
    The print '' in python2 was supported. But in python3,
    print '' was error. In python3, recommend using print()
    instead. This patch will fix it.
    
    Change-Id: I226461b0400023dc44238d9e5ee1ae2f2430de9e
    Closes-Bug: #1595773

commit 3ad003cf51151f8ce6dfc6c2c529206eda5f7b60
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Mon Jun 6 18:38:50 2016 +0100

Enable middleware to set metadata on object POST
    
    Adds a new form of system metadata for objects.
    
    Sysmeta cannot be updated by an object POST because
    that would cause all existing sysmeta to be deleted.
    Crypto middleware will want to add 'system' metadata
    to object metadata on PUTs and POSTs, but it is ok
    for this metadata to be replaced en-masse on every
    POST.
    
    This patch introduces x-object-transient-sysmeta-*
    that is persisted by object servers and returned
    in GET and HEAD responses, just like user metadata,
    without polluting the x-object-meta-* namespace.
    All headers in this namespace will be filtered
    inbound and outbound by the gatekeeper, so cannot
    be set or read by clients.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Co-Authored-By: Janie Richling <jrichli@us.ibm.com>
    
    Change-Id: I5075493329935ba6790543fc82ea6e039704811d

commit 029c2782ddfd933af20b697c00a46b1375e0f23e
Author: Christian Schwede <cschwede@redhat.com>
Date:   Mon Jun 27 14:18:43 2016 +0200

Add swiftbackmeup to associated projects
    
    Change-Id: I99f7a38d9b26605324408f1d200bf08da1e2772f

commit c953e84e28b9b17e16bde7dfbbbdabca7acded13
Author: liangjingtao <liang.jingtao@zte.com.cn>
Date:   Fri Jun 24 11:50:20 2016 +0800

Make string.letters PY3 compatible
    
    String.letters are removed in py3,use string.ascii_letters instead.
    
    Change-Id: I3c71b65b09b42dc954a3eb9e02894e5d3b12a3f4
    Closes-Bug: #1595786

commit 143b0eec92b723d11d5d1731ac2ae179e4d390e3
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 23 10:46:27 2016 -0700

MockMemcached cleanup
    
    * Break sendall's switch into different functions
    * Actually name some parameters
    * Raise errors on unexpected input
    * Consistently use tuples in self.cache
    
    Change-Id: I93aa33f83cdf6943a43f14a1868b1497bc7f4478

commit b923e0f8928d70b23c894db8d2d24bdfdc6d283d
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jun 23 13:34:51 2016 +0200

Python 3: dict.iteritems() and dict.itervalues()
    
    * Replace dict.itervalues() with dict.values(). The Python 3 dict
      type has no itervalues() method, the old itervalues() method was
      renamed to values().
    * Same change for dict.iteritems(), replaced with dict.items()
    * Exception: use six.itervalues() to yield on sock_data_by_port
    
    Using six.itervalues() and six.iteritems() would make the code less
    readable. The overhead of creating a temporary list is considered as
    negligible:
    http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
    
    Change-Id: Ifbe7faa16d419e7fe26f1fb464019b83c9171c45

commit a60096769c13ba8bd4c99a5ec516741f91305191
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jun 23 13:29:51 2016 +0200

Python 3: Fix basestring, long and StringIO
    
    * The basestring type was removed in Python 3: replace it with
      six.string_types.
    * Replace StringIO.StringIO with six.StringIO
    * Replace (int, long) with six.integer_types
    
    Change-Id: Ic0d443b0bdd00fb18452e79ffae07b9be0fa8116

commit fa7d80029b53391a7877aeb6438c98a45bab42a7
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Mon Jun 6 18:16:11 2016 +0100

Make container update override headers persistent
    
    Whatever container update override etag is sent to the object server
    with a PUT must be used in container updates for subsequent
    POSTs. Unfortunately the current container update override headers
    (x-backend-container-update-override-*) are not persisted with the
    object metadata so are not available when handling a POST.
    
    For EC there is an ugly hack in the object server to use the
    x-object-sysmeta-ec-[etag,size] values when doing a container update
    for a POST.
    
    With crypto, the encryption middleware needs to override the etag
    (possibly overriding the already overridden EC etag value) with an
    encrypted etag value. We therefore have a similar problem that this
    override value is not persisted at the object server.
    
    This patch introduces a new namespace for container override headers,
    x-object-sysmeta-container-update-override-*, which uses object
    sysmeta so that override values are persisted. This allows a general
    mechanism in the object server to apply the override values (if any
    have been set) from object sysmeta when constructing a container
    update for a PUT or a POST. Middleware should use the
    x-object-sysmeta-container-update-override-* namespace when setting
    container update overrides. Middleware should be aware that other
    middleware may have already set container override headers, in which
    case consideration should be given to whether any existing value should
    take precedence.
    
    For backwards compatibility the existing
    x-backend-container-update-override-* style headers are still
    supported in the object server for EC override values, and the ugly
    hack for EC etag/size override in POST updates remains in the object
    server. That allows an older proxy server to be used with an upgraded
    object server. The proxy server continues to use the
    x-backend-container-update-override-* style headers for EC values so
    that an older object server will continue to work with an upgraded
    proxy server.
    
    x-object-sysmeta-container-update-override-* headers take precedence
    over x-backend-container-update-override-* headers and the use of
    x-backend-container-update-override-* headers by middleware is
    deprecated.  Existing third party middleware that is using
    x-backend-container-update-override-* headers should be modified to
    use x-object-sysmeta-container-update-override-* headers in order to
    be compatible with other middleware such as encryption and to ensure
    that container updates during POST requests carry correct values. If
    targeting multiple versions of Swift object servers it may be
    necessary to send headers from both namespaces. However, in general it
    is recommended to upgrade all backend servers, then upgrade proxy
    servers before finally upgrading third party middleware.
    
    Co-Authored-By: Tim Burke <tim.burke@gmail.com>
    
    UpgradeImpact
    
    Change-Id: Ib80b4db57dfc2d37ea8ed3745084a3981d082784

commit 03b762e80a9b3d33ce13b8222f4cd2b549171c51
Author: Janie Richling <jrichli@us.ibm.com>
Date:   Mon Jun 6 17:19:48 2016 +0100

Support for http footers - Replication and EC
    
    Before this patch, the proxy ObjectController supported sending
    metadata from the proxy server to object servers in "footers" that
    trail the body of HTTP PUT requests, but this support was for EC
    policies only.  The encryption feature requires that footers are sent
    with both EC and replicated policy requests in order to persist
    encryption specific sysmeta, and to override container update headers
    with an encrypted Etag value.
    
    This patch:
    
    - Moves most of the functionality of ECPutter into a generic Putter
      class that is used for replicated object PUTs without footers.
    
    - Creates a MIMEPutter subclass to support multipart and multiphase
      behaviour required for any replicated object PUT with footers and
      all EC PUTs.
    
    - Modifies ReplicatedObjectController to use Putter objects in place
      of raw connection objects.
    
    - Refactors the _get_put_connections method and _put_connect_node methods
      so that more code is in the BaseObjectController class and therefore
      shared by [EC|Replicated]ObjectController classes.
    
    - Adds support to call a callback that middleware may have placed
      in the environ, so the callback can set footers. The
      x-object-sysmeta-ec- namespace is reserved and any footer values
      set by middleware in that namespace will not be forwarded to
      object servers.
    
    In addition this patch enables more than one value to be added to the
    X-Backend-Etag-Is-At header. This header is used to point to an
    (optional) alternative sysmeta header whose value should be used when
    evaluating conditional requests with If-[None-]Match headers.  This is
    already used with EC policies when the ECObjectController has
    calculated the actual body Etag and sent it using a footer
    (X-Object-Sysmeta-EC-Etag). X-Backend-Etag-Is-At is in that case set
    to X-Object-Sysmeta-Ec-Etag so as to point to the actual body Etag
    value rather than the EC fragment Etag.
    
    Encryption will also need to add a pointer to an encrypted Etag value.
    However, the referenced sysmeta may not exist, for example if the
    object was created before encryption was enabled. The
    X-Backend-Etag-Is-At value is therefore changed to support a list of
    possible locations for alternate Etag values. Encryption will place
    its expected alternative Etag location on this list, as will the
    ECObjectController, and the object server will look for the first
    object metadata to match an entry on the list when matching
    conditional requests. That way, if the object was not encrypted then
    the object server will fall through to using the EC Etag value, or in
    the case of a replicated policy will fall through to using the normal
    Etag metadata.
    
    If your proxy has a third-party middleware that uses X-Backend-Etag-Is-At
    and it upgrades before an object server it's talking to then conditional
    requests may be broken.
    
    UpgradeImpact
    
    Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
    Co-Authored-By: Thiago da Silva <thiago@redhat.com>
    Co-Authored-By: Samuel Merritt <sam@swiftstack.com>
    Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
    
    Closes-Bug: #1594739
    Change-Id: I12a6e41150f90de746ce03623032b83ed1987ee1

commit 4b1387968078bde6dcaeace5e390478d93b1780a
Author: Cloud User <cloud-user@bounce.cisco.com>
Date:   Wed May 4 02:00:53 2016 +0000

Adds migrated API reference files
    
    This brings in RST plus YAML files, migrated from the source for [0].
    
    The migration explanation is found on the openstack-dev mailing list [1].
    
    Project instruction is in the OpenStack Documentation Contributor Guide [2].
    
    A patch for publishing this source is in [3].
    
    The conf.py and the tox environment are standard across other projects.
    
    [0]http://developer.openstack.org/api-ref-objectstorage-v1.html
    [1]http://lists.openstack.org/pipermail/openstack-dev/2016-May/093765.html
    [2]http://docs.openstack.org/contributor-guide/api-guides.html
    [3]https://review.openstack.org/#/c/313015/
    
    Change-Id: Ifebc65b188c4f2ba35b61c0deae5ec24401df7f9

commit c0217a4845e2ea780dc4dcb61877e604bc488729
Author: Andreas Jaeger <aj@suse.com>
Date:   Tue Jun 21 07:56:35 2016 +0200

Update Sphinx version
    
    The api-ref document needs a newer sphinx version, allow a 1.2 Sphinx
    version to be used - like it's used in global-requirements.txt.
    
    Change-Id: I9183cc56753fbe7e41206c6a9081899df5c3919a
    Needed-By: Ifebc65b188c4f2ba35b61c0deae5ec24401df7f9

commit 5885d97b7df787ccd64777174b59c48e6dbfaa06
Author: John Dickinson <me@not.mn>
Date:   Fri Jun 17 12:24:57 2016 -0700

added note to testFileSizeLimit functional test
    
    Change-Id: I0323ff2511506c354db3416f1b37ede772acaedb

commit da4a59f8e276e80764b58f9c52d6a3c2bca06782
Author: Or Ozeri <oro@il.ibm.com>
Date:   Thu Jun 16 11:14:14 2016 +0300

pickle_async_update should create tmp_dir
    
    While creating a probe test for the expirer daemon, I found
    the following error scenario:
    
    1. Introduce a new object server. Initially it doesn't have a tmp_dir.
    2. Have the object-replicator replicate some objects, one of them
        with an expiration (X-Delete-At).
    3. Send a DELETE request for the expired object.
    
    While beginning to process the DELETE request, the fresh
    object server still doesn't have a tmp_dir created.
    Since the object has an old expiration value, the object server
    will first call "delete_at_update", before creating a tombstone.
    delete_at_update then must create an async_pending,
    which will lead to an IO error, since tmp_dir doesn't exist.
    
    As said, I have witnessed this in practice in the probe test I wrote
    at https://review.openstack.org/#/c/326903/.
    
    This patch changes pickle_async_update behavior to create
    tmp_dir, in case it doesn't exist.
    
    Change-Id: I88b0e5f75a2a28d6880694ff327ac2763c816d24

commit 928c4790ebce3782f42d239faa9758941a8dd296
Author: Alistair Coles <alistair.coles@hpe.com>
Date:   Tue Jun 7 13:41:55 2016 +0100

Refactor tests and add tests
    
    Relocates some test infrastructure in preparation for
    use with encryption tests, in particular moves the test
    server setup code from test/unit/proxy/test_server.py
    to a new helpers.py so that it can be re-used, and adds
    ability to specify additional config options for the
    test servers (used in encryption tests).
    
    Adds unit test coverage for extract_swift_bytes and functional
    test coverage for container listings. Adds a check on the content
    and metadata of reconciled objects in probe tests.
    
    Change-Id: I9bfbf4e47cb0eb370e7a74d18c78d67b6b9d6645

commit 29b8d2da20bba3782b2182d01350f16e3db44263
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Tue Jun 14 00:52:24 2016 -0700

Avoid docs warning: Duplicate explicit target name
    
    When we add two (or more than) different links as same target name,
    we need to set two underscore[1] instead of one. This avoids "Duplicate
    explicit target name" warnings.
    
    1:
    http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#hyperlink-references
    
    Change-Id: I8a493e7a1deeece33ee1b3fb3f5c848f3cc31d06

commit 3944d820387f08372c1a29444f4af7d8e6090ae9
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Jun 14 16:10:16 2016 -0700

Catch AttributeError less often
    
    I always get tripped up when I'm editing code that catches
    AttributeError and does something with it. I'll type "req.emthod" or
    something, and next thing I know I'm getting 405s in all my unit
    tests. This diff removes some places where we catch AttributeError
    (sometimes, having deliberately thrown it only one line before) so
    that typos can crash the way Guido intended.
    
    Change-Id: I2f7586f96b41a97e6ae254efc83218b3b5c6cc9e

commit 65a9a6d21b81a000effd911cab5613a3fa6a784e
Author: Christian Schwede <cschwede@redhat.com>
Date:   Tue Jun 7 11:27:56 2016 +0000

Add simple multiple server type test
    
    Ensures that swift-recon actually gathers data from multiple
    server types if more than one is given on the command line.
    
    Change-Id: I4017b82fb044265ec117df01e14968752df02201

commit 6c9a1899a182d58232a95af92dae168df9e131d5
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Tue May 31 20:53:03 2016 +0000

Adds region as a search-value in manpage
    
    This patch adds region and example related to it in the man page
    of swift-ring-builder.
    
    Change-Id: I12ceab4e41240240cf2daa77dad94729dd1fd76d

commit 7a50972104095478c91a477f5c5499dda4372711
Author: John Dickinson <me@not.mn>
Date:   Thu Jun 9 11:22:37 2016 -0700

update .gitreview
    
    Change-Id: I9593e453891c137fd430a44306e17268ba45fd12

commit f6b0b75a25ef970e8bad67f554c82df676e0d172
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed Jun 8 10:01:39 2016 +0000

Make test_ringbuilder less brittle
    
    If one has an object.builder file in the current directory and runs
    test_ringbuilder, it will fail with an irritating error. That's because
    test_use_ringfile_as_builderfile doesn't use self.tmpfile, but
    object.builder - and that one might exist in the local directory.
    
    This patch changes this, using self.tmpfile as argument name.
    
    Closes-Bug: 1590356
    Change-Id: I4b3287a36e8a5e469eb037128427dc7867910e53

commit 334140a543cb16aa4c7b01b1f4371fc4e982032f
Author: Mohit Motiani <mohit.motiani@intel.com>
Date:   Tue Jun 7 19:08:50 2016 +0000

Removed whitespaces from swift-ring-builder manpage
    
    Change-Id: I25cf69f8d963ba84df4c59129d72ee39ec341bd3

commit fcb6e4cd3aa6896976733c57c683592358e4c6f0
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Thu Jul 2 00:35:02 2015 -0700

Last-Modified header support on HEAD/GET container
    
    This patch enables to show a x-put-timestamp as
    a last-modified header in container-server.
    
    Note that the last-modified header will be changed only when a
    request for container (PUT container or POST container) comes into
    Swift. i.e. some requests for objects (e.g. PUT object, POST object)
    will never affect the last-modified value but only when using
    python-swiftclient like as "swift upload", the last-modified will
    be close to the upload time because python-swiftclient will make
    a PUT container request for "swift upload" each time.
    
    Change-Id: I9971bf90d24eee8921f67c02b7e2c80fd8995623

commit 66c77e049dc60ec012fb34a45ef9531f11ea906a
Author: Sivasathurappan Radhakrishnan <siva.radhakrishnan@intel.com>
Date:   Fri Feb 19 21:54:12 2016 +0000

Added unit test cases for cli/recon.py
    
    Added unit test cases to cover code paths of umount_check
    and async_check function.
    
    Change-Id: I236d24b1b22ce244b3fb9546ff31223426edb8ed

commit ef942dadebc038a82d9198b8048244c980cd2c24
Author: Peter Lisák <peter.lisak@firma.seznam.cz>
Date:   Fri Nov 13 13:56:13 2015 +0100

Call swift-recon with more than one server type
    
    Examples:
    * get auditor stats from account and container server
    swift-recon account container --auditor
    * perform all checks on all servers
    swift-recon account container object --all
    
    Change-Id: I05bb7c785e6a4d80969d90448f1b9e0fd08dae86
    Co-Authored-By: Ondřej Nový <ondrej.novy@firma.seznam.cz>

commit efdf123a402b373a2d572e777f535c21241b4bb8
Author: Andy McCrae <andy.mccrae@gmail.com>
Date:   Thu May 12 15:46:24 2016 +0100

[Docs] Document prevention of disk full scenarios
    
    Adds section to detail how to prevent disk full scenarios from
    occurring.
    
    Change-Id: Iafb4a47fa4892f6067252f3a80de87cd76506a40

commit b72141979f8766f411d2fea8fe8856b157bc9b91
Author: Yatin Kumbhare <yatinkumbhare@gmail.com>
Date:   Mon Dec 14 17:04:34 2015 +0530

Fixed inconsistencies in docstrings
    
    Fixed :returns:, and few one-line comments as per
    the docstring guide lines.
    http://docs.openstack.org/developer/hacking/#docstrings
    
    Change-Id: I36ecf4faf5b49e070c13eb6324841ebcf442524f

tags:

added: in-feature-hummingbird

OpenStack Object Storage (swift)

POST to DLO squashes data without fast-POST

Bug Description

Duplicates of this bug

Other bug subscribers

Remote bug watches