commit 03e8ab7171580f3fb93131e9ccfb69fd35364672
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 17:15:54 2017 -0700
functests: don't crash if no second account
In test.functional.test_object.TestObject.setUp, we create a container
in account 2. However, if we've only got one account, we don't skip
this class, resulting in a TypeError down in requests somewhere and a
stack trace. Since we're using account 2 in setup, we should skip the
tests if account 2 is not configured.
commit eaea0c49332273085b17852e22582940ab059006
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 16:58:41 2017 -0700
Skip cross-account-copy functest if only one account
This looks like a case of copy-paste-itis. The cross-account-copy
functest is skipped if we have no test accounts configured, but not if
we have only one.
commit dc8da5bb194d0a544fb8065aa9a4c7484f605715
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 10:45:12 2017 -0700
Use "poll" or "selects" Eventlet hub for all Swift daemons.
Previously, Swift's WSGI servers, the object replicator, and the
object reconstructor were setting Eventlet's hub to either "poll" or
"selects", depending on availability. Other daemons were letting
Eventlet use its default hub, which is "epoll".
In any daemons that fork, we really don't want to use epoll. Epoll
instances end up shared between the parent and all children, and you
get some awful messes when file descriptors are shared.
Here's an example where two processes are trying to wait on the same
file descriptor using the same epoll instance, and everything goes
wrong:
This primarily affects the container updater and object updater since
they fork. I've decided to change the hub for all Swift daemons so
that we don't add multiprocessing support to some other daemon someday
and suffer through this same bug again.
This problem was made more apparent by commit 6d16079, which made our
logging mutex use file descriptors. However, it could have struck on
any shared file descriptor on which a read or write returned EAGAIN.
commit 37f23b072ee7a655ab9072f1d0bece0919dec61b
Author: Samuel Merritt <email address hidden>
Date: Mon Oct 9 17:31:30 2017 -0700
Allow SLOs to have zero-byte last segments.
Since we used to allow zero-byte last segments but now we don't, it
can be difficult to deal with some old SLO manifests.
Imagine you're writing some code to sync objects from Swift cluster A
to Swift cluster B. You start off with just a GET from A piped into a
PUT to B, and that works great until you hit a SLO manifest and B
won't accept a 500GB object. So, you write some code to detect SLO
manifests, sync their segments, then take the JSON manifest
(?multipart-manifest=get) and sync *that* over.
Now, life is good... until one day you get an exception notification
that there's this manifest on cluster A that cluster B won't
accept. Turns out that, back when Swift would take zero-byte final
segments on SLOs (before commit 7f636a5), someone uploaded such a SLO
to cluster A. Now, however, zero-byte final segments are invalid, so
that SLO that exists over in cluster A can't just be copied to cluster
B.
A little coding later, your sync tool detects zero-byte final segments
and removes them when copying a manifest. But now your ETags don't
match between clusters, so you have to figure out some way to deal
with that, and so you put it in metadata, but then you realize that
your syncer might encounter a SLO which contains a sub-SLO which has a
zero-byte final segment, and it's right about then that you start
thinking about giving up on programming and getting a job as an
elevator mechanic.
This commit makes life easier for developers of such applications by
allowing SLOs to have zero-byte segments again.
Reviewed: https:/ /review. openstack. org/513730 /git.openstack. org/cgit/ openstack/ swift/commit/ ?id=bb064f0b30e c083cec85e8371f e12824804779cf
Committed: https:/
Submitter: Zuul
Branch: feature/s3api
commit 0bca0d4d2bf95a3 7e4a403457ee5c7 db4c0b6290
Author: Pete Zaitcev <email address hidden>
Date: Wed Oct 18 18:15:11 2017 -0500
Be more tolerant of exception messages from sqlite
Parsing the text of error messages was always perilous, perhaps.
Either way, this should fix the problem of test_get() failing.
Change-Id: I929c2f6e606242 41075a292d020c5 707ea0ff1c3
commit e001c02ff938d9a ea560970fa70f3d 8884a8ea33
Author: Tim Burke <email address hidden>
Date: Tue Oct 17 22:49:39 2017 +0000
Stop logging tracebacks on bad xLOs
The error messages alone should provide plenty enough information.
Plus, running functional tests really shouldn't cause tracebacks.
Also, tighten up tests for log messages.
Change-Id: I55136484d342af 756fa153d971dcb 9159a435f13
commit 356b110229ee5e5 be93ce1ec60ed2d e8f75090bf
Author: Samuel Merritt <email address hidden>
Date: Wed Oct 18 15:09:12 2017 -0700
Remove swift-temp-url man page.
Commit 250da37a removed bin/swift-temp-url, but left the man page.
Change-Id: Ic518f23678e3c3 134f02a46a51a2b cb90d92bdc2
commit 1aecd1dfc61988f 12d4cdfda7110dd 648133e4cf
Author: Alistair Coles <email address hidden>
Date: Wed Oct 18 10:52:52 2017 +0100
tighten up drop_privileges unit tests
add more assertions about args that are passed to
os module functions
Related-Change: Ida15e72ae4ecdc 2d6ce0d37bd99c2 d86bd4e5ddc c9930cd54008533 de2917157f4
Change-Id: Iee483274aff37f
commit e6cf9b4758612f3 8f7317c52423f31 7b9a11bbc9
Author: Samuel Merritt <email address hidden>
Date: Tue Oct 17 15:16:43 2017 -0700
Fix some spelling in a docstring
Change-Id: I6b32238da33818 48ae56aed1c570d 299be72473e
commit 646f7507a1f5cf4 79b8d2023645009 de0a28cd79
Author: Tim Burke <email address hidden>
Date: Tue Oct 17 21:17:57 2017 +0000
Quiet test output when running test_utils.py in isolation
Change-Id: I4cf85d3cd5a204 24e9bbbdf021312 0b3c3d4b837
commit 0f91c862e19bb8d 81c20d03e65e6e6 657f7265dc
Author: Corey Bryant <email address hidden>
Date: Tue Oct 17 15:04:45 2017 -0400
Drop group comparison from drop_privileges test
Drop the group comparison from drop_privileges test as it isn't
valid since os.setgroups() is mocked.
Change-Id: Ida15e72ae4ecdc 2d6ce0d37bd99c2 d86bd4e5ddc
Closes-Bug: #1724342
commit 5438fe7d9bcb7dc 99118c12bdf1ccf d663390b08
Author: Samuel Merritt <email address hidden>
Date: Fri Oct 13 17:03:18 2017 -0700
Clean up some leftover Python 2.6-isms.
Change-Id: I24f0aa3bf7314a 669c4cf44eadc46 e874d6803f1
commit 250da37a7b279b0 5a1ac8954b506ad d1661f194d
Author: Tim Burke <email address hidden>
Date: Fri Oct 13 23:27:11 2017 +0000
Remove swift-temp-url script
This has been deprecated since Swift 2.10.0 (Newton) including a
message that it would go away. Let's actually remove it.
Change-Id: I7d3659761c7111 9363ff2c0c750e3 7b4c6374a39 db4845b02d1b586 99edaa39356
Related-Change: Ifa8bf636f20f82
commit c118059719ded85 381e9b134b091d0 4a7b5a3955
Author: Tim Burke <email address hidden>
Date: Mon Sep 11 22:08:12 2017 +0000
Respond 400 Bad Request when Accept headers fail to parse
Change-Id: I6eb4e4bca95e2e e4fecdb703394cb 2419737922d
Closes-Bug: 1716509
commit 03e8ab7171580f3 fb93131e9ccfb69 fd35364672
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 17:15:54 2017 -0700
functests: don't crash if no second account
In test.functional .test_object. TestObject. setUp, we create a container
in account 2. However, if we've only got one account, we don't skip
this class, resulting in a TypeError down in requests somewhere and a
stack trace. Since we're using account 2 in setup, we should skip the
tests if account 2 is not configured.
Change-Id: I569d98baf071d2 dce7cf34a953807 0f00afda388
commit eaea0c493322730 85b17852e225829 40ab059006
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 16:58:41 2017 -0700
Skip cross-account-copy functest if only one account
This looks like a case of copy-paste-itis. The cross-account-copy
functest is skipped if we have no test accounts configured, but not if
we have only one.
Change-Id: Ifbefdd9aeb98e3 d02c536e9d29759 f86ec9af6a1
commit dc8da5bb194d0a5 44fb8065aa9a4c7 484f605715
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 10:45:12 2017 -0700
Use "poll" or "selects" Eventlet hub for all Swift daemons.
Previously, Swift's WSGI servers, the object replicator, and the
object reconstructor were setting Eventlet's hub to either "poll" or
"selects", depending on availability. Other daemons were letting
Eventlet use its default hub, which is "epoll".
In any daemons that fork, we really don't want to use epoll. Epoll
instances end up shared between the parent and all children, and you
get some awful messes when file descriptors are shared.
Here's an example where two processes are trying to wait on the same
file descriptor using the same epoll instance, and everything goes
wrong:
[proc A] epoll_ctl(6, EPOLL_CTL_ADD, 3, ...) = 0
[proc B] epoll_ctl(6, EPOLL_CTL_ADD, 3, ...) = -1 EEXIST (File exists)
[proc B] epoll_wait(6, ...) = 1
[proc B] epoll_ctl(6, EPOLL_CTL_DEL, 3, ...) = 0
[proc A] epoll_wait(6, ...)
This primarily affects the container updater and object updater since
they fork. I've decided to change the hub for all Swift daemons so
that we don't add multiprocessing support to some other daemon someday
and suffer through this same bug again.
This problem was made more apparent by commit 6d16079, which made our
logging mutex use file descriptors. However, it could have struck on
any shared file descriptor on which a read or write returned EAGAIN.
Change-Id: Ic2c1178ac918c8 8b0b901e581eb4f ab3b2666cfe
Closes-Bug: 1722951
commit 37f23b072ee7a65 5ab9072f1d0bece 0919dec61b
Author: Samuel Merritt <email address hidden>
Date: Mon Oct 9 17:31:30 2017 -0700
Allow SLOs to have zero-byte last segments.
Since we used to allow zero-byte last segments but now we don't, it
can be difficult to deal with some old SLO manifests.
Imagine you're writing some code to sync objects from Swift cluster A -manifest= get) and sync *that* over.
to Swift cluster B. You start off with just a GET from A piped into a
PUT to B, and that works great until you hit a SLO manifest and B
won't accept a 500GB object. So, you write some code to detect SLO
manifests, sync their segments, then take the JSON manifest
(?multipart
Now, life is good... until one day you get an exception notification
that there's this manifest on cluster A that cluster B won't
accept. Turns out that, back when Swift would take zero-byte final
segments on SLOs (before commit 7f636a5), someone uploaded such a SLO
to cluster A. Now, however, zero-byte final segments are invalid, so
that SLO that exists over in cluster A can't just be copied to cluster
B.
A little coding later, your sync tool detects zero-byte final segments
and removes them when copying a manifest. But now your ETags don't
match between clusters, so you have to figure out some way to deal
with that, and so you put it in metadata, but then you realize that
your syncer might encounter a SLO which contains a sub-SLO which has a
zero-byte final segment, and it's right about then that you start
thinking about giving up on programming and getting a job as an
elevator mechanic.
This commit makes life easier for developers of such applications by
allowing SLOs to have zero-byte segments again.
Change-Id: Ia37880bbb435e2 69ec53b2963eb1b 9121696d479
commit a959d24bf573365 0c6005412382fe2 4c65897c8b
Author: Alistair Coles <email address hidden>
Date: Tue Aug 15 12:37:33 2017 +0100
Document keystone role element in container ACL
The use of a keystone role name in container ACLs is supported
and tested. This patch adds documentation.
[1] https:/ /github. com/openstack/ swift/blob/ fb3d01a974fb7df 8cfadc56ff15bdc 04b3c90759/ swift/common/ middleware/ keystoneauth. py#L491- L497 common. middleware. test_keystoneau th.TestAuthoriz e.test_ authorize_ succeeds_ for_user_ role_in_ roles
[2] test.unit.
Change-Id: I77df27393a10f1 d8c5a43161fdd4e b08be632566
Closes-Bug: #1705300