Update Container-Sync Docs for internal_client.conf
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Object Storage (swift) |
Fix Released
|
Medium
|
Kazuhiro MIYAHARA | ||
| openstack-manuals |
Won't Fix
|
Medium
|
Alexandra Settle | ||
Bug Description
https:/
commit e440d6aed8a4084
Author: Kota Tsuyuzaki <email address hidden>
Date: Wed Apr 15 11:25:13 2015 -0700
Fix best response to return correct status
Current best response could return "503 Internal Server Error".
However, "503" means "Service Unavailable". (The status int of
Internal Server Error is 500)
This patch fixes the response status as "503 Service Unavailable"
Change-Id: I88b8c52c26b19e
commit 57011d5699d49a4
Author: Ricardo Ferreira <email address hidden>
Date: Thu Mar 12 23:18:33 2015 +0000
More user-friendly output for object metadata
Split out system, user and other metadata in swift-object-info. Print
every position line by line instead of raw dict representation, so it
would be easier to parse with tools such as grep.
Co-Authored-By: Ricardo Ferreira <email address hidden>
Co-Authored-By: Kamil Rykowski <email address hidden>
Change-Id: Ia78da518c18f7e
Closes-Bug: #1428866
commit a162c2bdd7be12d
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Apr 16 06:06:35 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https:/
Change-Id: I48ba06f4801ff2
commit 52b102163e48dc0
Author: Clay Gerrard <email address hidden>
Date: Wed Apr 15 15:31:06 2015 -0700
Don't apply the wrong Etag validation to rebuilt fragments
Because of the object-server's interaction with ssync sender's
X-Backend-
pushed unmodified to another node it's ETag value is duped into the
recieving ends metadata as Etag. This interacts poorly with the
reconstructor's RebuildingECDis
time the ETag of the fragment archive being rebuilt.
Don't send the Etag from the local source fragment archive being used as
the basis for the rebuilt fragent archive's metadata along to ssync.
Change-Id: Ie59ad93a67a7f4
commit 46bd6716ffae28a
Author: Kota Tsuyuzaki <email address hidden>
Date: Tue Apr 14 23:22:14 2015 -0700
Small minor refactor on ec diskfile
To be more helpful, add an inline comment and remove
unnecessary assignment.
Change-Id: Ia9c6993dfa03c2
commit 193fe9a5f906a23
Author: Lorcan <email address hidden>
Date: Wed Apr 15 11:32:32 2015 +0100
Update swift-recon doc with more options
The swit-recon tool has had several functional additions
added recently but not all of these have been added to the docs.
This change adds the following options to the manpages:
-
-
--sockstat
--driveaudit
--region
--timeout
Also fixes a typo on line 78 (cop -> copy)
Change-Id: Id083b32a60473a
commit 663ccd4e7ad50f6
Author: Christian Schwede <email address hidden>
Date: Tue Apr 14 10:52:36 2015 +0000
More tests for swift recon
Change-Id: I8d568c0f6fbe1c
commit c8e943c916c6e2e
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Apr 15 06:07:23 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https:/
Change-Id: I15b1d475d157bf
commit 5ef2e9ec00b1498
Author: Clay Gerrard <email address hidden>
Date: Tue Apr 14 15:24:58 2015 -0700
fixup random test failure
Change-Id: I053cec441623e7
commit 86920f3dbe43a03
Author: John Dickinson <email address hidden>
Date: Mon Mar 23 12:00:28 2015 -0700
2.3.0 authors and changelog updates
Change-Id: I5a46437a718e2a
commit 4aba2fbb25edf89
Author: Clay Gerrard <email address hidden>
Date: Fri Mar 27 15:50:38 2015 -0700
Check if REST API version is valid
Swift doesn't check if the used API version is valid. Currently there
is only one valid REST API version, but that might change in the
future.
This patch enforces "v1" or "v1.0" as the version string when accessing
account, containers and objects.
The list of accepted version strings can be manually overridden using a
comma-separated list in swift.conf to make this backward-
The constraint loader has been modified slightly to accept strings as
well as integers.
Any request to an account, container, and object which does not
provide the correct version string will get a 400 BadRequest response.
The allowed api versions are by default excluded from /info.
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: John Dickinson <email address hidden>
Closes Bug #1437442
Change-Id: I5ab6e236544378
commit e16df14a734757f
Author: Christian Schwede <email address hidden>
Date: Sun Apr 12 22:35:56 2015 +0200
Add test for swift_recon.
Change-Id: I4cab7aa6df3f0e
commit 66fb207fcb28888
Author: Lorcan <email address hidden>
Date: Tue Mar 24 13:54:00 2015 +0000
Add lowest option to swift-recon disk usage output
Currently there is a "--top" option when running swift-recon for
disk usage stats. This option lists the x disks with the highest
disk usage in descending order.
This feature adds a "--lowest" option which does the opposite and
lists the y disks with lowest disk usage in ascending order.
Have also updated the docs section with --top and --lowest options
Change-Id: Ic15d407fe010a3
commit b76d8c5156c65e3
Author: Mike Fedosin <email address hidden>
Date: Wed Apr 8 15:00:27 2015 +0300
Fix broken test_policy_
Since the existing check depends on how quickly the test
passes, it causes to the unpredictable behavior and may break
the test.
This commit removes the unnecessary dependency on the
current time and leaves only the check that timestamp is
greater than 0.
Change-Id: I3321e9aba0df52
Closes-Bug: #1441599
commit 26bff69cb5c5f86
Author: Andreas Jaeger <email address hidden>
Date: Wed Apr 15 09:43:26 2015 +0200
Fix untranslatable string
The word "number" is not marked for translation, we cannot just add
it this way to a string.
Instead create two separate strings.
Change-Id: I0396f1d7405987
Closes-Bug: #1444327
commit 749bdac1df35713
Author: Hisashi Osanai <email address hidden>
Date: Mon Mar 23 13:47:22 2015 +0900
Refactor the getting roles logic in _keystone_identity
The list_from_csv in swift.common.utils is used from the composite
auth token support and this method has been used in
_integral_
_keystone_
Change-Id: I3d72a34e6fc21f
commit 8a1453e61e95956
Author: Donagh McCabe <email address hidden>
Date: Wed Feb 25 13:54:55 2015 +0000
Support HTTP_X_
Keystone's auth_token now supports delay denial when
a service token is invalid. This change handles
the consequences for determining 401 vs 403 responses.
Related-Bug: #1422389
Change-Id: I90ba062e357ccc
commit d3213fb1fe00ae6
Author: Christian Schwede <email address hidden>
Date: Tue Mar 31 08:51:18 2015 +0000
Check if device name is valid when adding to the ring
Currently device names can be empty or start and/or end with spaces.
This can create unexpected results, for example these three commands
are all valid:
swift-
swift-
swift-
This patch validates device names and prevents empty names or names
starting and/or ending with spaces.
Also fixed the test "test_warn_at_risk" - the test passed if the
exception was not raised.
Closes-Bug: 1438579
Change-Id: I811b0eae7db503
commit 3c419dfb12f19d3
Author: Richard Hawkins <email address hidden>
Date: Sat Feb 14 10:19:16 2015 -0600
Add additional func tests for TempURLs
Change-Id: I25097c87d0a06c
commit 8f5d4d24557887b
Author: Paul Luse <email address hidden>
Date: Thu Jul 3 15:59:33 2014 -0700
Erasure Code Documentation
This patch adds all the relevant EC documentation to
the source tree. Notable additions are:
- Updated SAIO documentation
- Updates to existing swift documentation; and
- Erasure Coding overview
Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: John Dickinson <email address hidden>
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Tushar Gohad <email address hidden>
Co-Authored-By: Samuel Merritt <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
Change-Id: I0403016a4bb7da
Implements: blueprint swift-ec
Signed-off-by: Thiago da Silva <email address hidden>
commit 647b66a2ce4c85c
Author: paul luse <email address hidden>
Date: Tue Oct 28 09:51:06 2014 -0700
Erasure Code Reconstructor
This patch adds the erasure code reconstructor. It follows the
design of the replicator but:
- There is no notion of update() or update_deleted().
- There is a single job processor
- Jobs are processed partition by partition.
- At the end of processing a rebalanced or handoff partition, the
And various ssync changes such as the addition of reconstruct_fa()
function called from ssync_sender which performs the actual
reconstruction while sending the object to the receiver
Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: John Dickinson <email address hidden>
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Tushar Gohad <email address hidden>
Co-Authored-By: Samuel Merritt <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
blueprint ec-reconstructor
Change-Id: I7d15620dc66ee6
commit b2189ef47ae08c3
Author: Clay Gerrard <email address hidden>
Date: Fri Apr 3 16:23:14 2015 -0700
Fix account reaper for > 3 replicas
There's a pre-existing IndexError in the pop from the container node
list in reaper's reap_object method for object rings with a replica
count greater than the container replica count. Which is more likely
on EC storage policies.
When making the backend direct delete requests to the nodes once the
container node's list is exhausted the generic exception handler logs
the error and breaks out of any other backend object requests - but
the reaper marches forward and eventually the tombstones are
replicated.
This change just cycles the container headers across all the nodes -
which seems reasonable enough - but could certainly garner
bikeshedding.
Change-Id: I5897d00b0a8c1e
commit 61a9d35fd58381b
Author: Yuan Zhou <email address hidden>
Date: Wed Dec 24 11:52:34 2014 +0800
Update container sync to use internal client
This patch changes container sync to use Internal Client instead
of Direct Client.
In the current design, container sync uses direct_get_object to
get the newest source object(which talks to storage node directly).
This works fine for replication storage policies however in
erasure coding policies, direct_get_object would only return part
of the object(it's encoded as several pieces). Using Internal
Client can get the original object in EC case.
Note that for the container sync put/delete part, it's working in
EC since it's using Simple Client.
Signed-off-by: Yuan Zhou <email address hidden>
DocImpact
Change-Id: I91952bc9337f35
commit decbcd24d41d636
Author: Samuel Merritt <email address hidden>
Date: Wed Oct 22 13:18:34 2014 -0700
Foundational support for PUT and GET of erasure-coded objects
This commit makes it possible to PUT an object into Swift and have it
stored using erasure coding instead of replication, and also to GET
the object back from Swift at a later time.
This works by splitting the incoming object into a number of segments,
erasure-coding each segment in turn to get fragments, then
concatenating the fragments into fragment archives. Segments are 1 MiB
in size, except the last, which is between 1 B and 1 MiB.
+==
| object data |
+==
| | |
v v v
+==
| segment 1 | | segment 2 | ... | segment N |
+==
| |
| |
v v
| pyeclib | | pyeclib | ...
| |
| |
+--> fragment A-1 +--> fragment A-2
| |
| |
| |
| |
| |
+--> fragment B-1 +--> fragment B-2
| |
| |
... ...
Then, object server A gets the concatenation of fragment A-1, A-2,
..., A-N, so its .data file looks like this (called a "fragment archive"):
+==
| fragment A-1 | fragment A-2 | ... | fragment A-N |
+==
Since this means that the object server never sees the object data as
the client sent it, we have to do a few things to ensure data
integrity.
First, the proxy has to check the Etag if the client provided it; the
object server can't do it since the object server doesn't see the raw
data.
Second, if the client does not provide an Etag, the proxy computes it
and uses the MIME-PUT mechanism to provide it to the object servers
after the object body. Otherwise, the object would not have an Etag at
all.
Third, the proxy computes the MD5 of each fragment archive and sends
it to the object server using the MIME-PUT mechanism. With replicated
objects, the proxy checks that the Etags from all the object servers
match, and if they don't, returns a 500 to the client. This mitigates
the risk of data corruption in one of the proxy --> object connections,
and signals to the client when it happens. With EC objects, we can't
use that same mechanism, so we must send the checksum with each
fragment archive to get comparable protection.
On the GET path, the inverse happens: the proxy connects to a bunch of
object servers (M of them, for an M+K scheme), reads one fragment at a
time from each fragment archive, decodes those fragments into a
segment, and serves the segment to the client.
When an object server dies partway through a GET response, any
partially-
back to the nearest fragment boundary, and the GET is retried with the
next object server.
GET requests for a single byterange work; GET requests for multiple
byteranges do not.
There are a number of things _not_ included in this commit. Some of
them are listed here:
* multi-range GET
* deferred cleanup of old .data files
* durability (daemon to reconstruct missing archives)
Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: John Dickinson <email address hidden>
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Tushar Gohad <email address hidden>
Co-Authored-By: Paul Luse <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
Change-Id: I9c13c03616489f
commit b1eda4aef8a2289
Author: Samuel Merritt <email address hidden>
Date: Tue Sep 16 18:40:41 2014 -0700
Allow sending object metadata after data
This lets the proxy server send object metadata to the object server
after the object data. This is necessary for EC, as it allows us to
compute the etag of the object in the proxy server and still store it
with the object.
The wire format is a multipart MIME document. For sanity during a
rolling upgrade, the multipart MIME document is only sent to the
object server if it indicates, via 100 Continue header, that it knows
how to consume it.
Example 1 (new proxy, new obj server):
proxy: PUT /p/a/c/o
obj: 100 Continue
proxy: --MIMEmimeMIMEm
Example2: (new proxy, old obj server)
proxy: PUT /p/a/c/o
obj: 100 Continue
proxy: <obj body>
Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: John Dickinson <email address hidden>
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Tushar Gohad <email address hidden>
Co-Authored-By: Paul Luse <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
Change-Id: Id38f7e93e3473f
commit fa89064933fefa3
Author: Alistair Coles <email address hidden>
Date: Tue Mar 17 08:32:57 2015 +0000
Per-policy DiskFile classes
Adds specific disk file classes for EC policy types.
The new ECDiskFile and ECDiskFileWriter classes are used by the
ECDiskFileM
ECDiskFileM
EC_POLICY type policies.
Refactors diskfile tests into BaseDiskFileMixin and BaseDiskFileMan
classes which are then extended in subclasses for the legacy
replication
Refactor to prefer use of a policy instance reference over a policy_index
int to refer to a policy.
Add additional verification to DiskFileManager
device root with common.
disabled for use in on a virtual swift-all-in-one.
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: John Dickinson <email address hidden>
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Tushar Gohad <email address hidden>
Co-Authored-By: Paul Luse <email address hidden>
Co-Authored-By: Samuel Merritt <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
Change-Id: I22f915160dc67a
commit dd9d97458ea0070
Author: John Dickinson <email address hidden>
Date: Fri Mar 20 10:17:25 2015 +0000
Prevent unauthorized delete in versioned container
An authenticated user can delete the most recent version of any
versioned object who's name is known if the user has listing access
to the x-versions-location container. Only Swift setups with
allow_version setting are affected.
This patch closes this bug, tracked as CVE-2015-1856
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Alistair Coles <email address hidden>
Closes-Bug: 1430645
Change-Id: Ibacc7413afe7cb
commit a7078293349a01a
Author: Clay Gerrard <email address hidden>
Date: Tue Mar 31 22:35:37 2015 -0700
Update test infrastructure
* Get FakeConn ready for expect 100 continue
* Use debug_logger more and with better interfaces
* Fix patch_policies to be less annoying
Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: Tushar Gohad <email address hidden>
Co-Authored-By: Paul Luse <email address hidden>
Co-Authored-By: Samuel Merritt <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
Change-Id: I28c0a3539d994c
commit ed5406628884432
Author: Tushar Gohad <email address hidden>
Date: Mon Jun 30 11:14:28 2014 -0700
Add support for policy types, 'erasure_coding' policy
This patch extends the StoragePolicy class for non-replication storage
policies, the first one being "erasure coding".
Changes:
- Add 'policy_type' support to BaseStoragePolicy class
- Disallow direct instantiation of BaseStoragePolicy class
- Subclass BaseStoragePolicy
- "StoragePolicy":
. Replication policy, default
. policy_type = 'replication'
- "ECStoragePolicy":
. Erasure Coding policy
. policy_type = 'erasure_coding'
. Private member variables
ec_type (EC backend),
data split into after erasure coding operation),
. Private methods
EC specific attributes and ring validator methods.
- Swift will use PyECLib, a Python Erasure Coding library, for
erasure coding operations. PyECLib is already an approved
OpenStack core requirement.
(https:/
- Add test cases for
- 'policy_type' StoragePolicy member
- policy_type == 'erasure_coding'
DocImpact
Co-Authored-By: Alistair Coles <email address hidden>
Co-Authored-By: Thiago da Silva <email address hidden>
Co-Authored-By: Clay Gerrard <email address hidden>
Co-Authored-By: Paul Luse <email address hidden>
Co-Authored-By: Samuel Merritt <email address hidden>
Co-Authored-By: Christian Schwede <email address hidden>
Co-Authored-By: Yuan Zhou <email address hidden>
Change-Id: Ie0e09796e3ec45
Implements: blueprint ec-proxy-work
commit 16ee994c1ea4ddc
Author: Christian Schwede <email address hidden>
Date: Wed Feb 18 10:17:23 2015 +0000
Set connection timeout in container sync
Container sync might get stuck without a connection timeout if the remote proxy
is not responding.
This patch sets a default timeout of 5.0 seconds for the connection attempt. The
value is much higher than other connection timeouts inside Swift (0.5); however
there might be a much higher latency to the remote peer, thus playing it safe.
There is also a retry if the attempt timed out.
Note that this setting only applies to the connection request itself. Setting
this timeout does not apply when the remote proxy goes away during a request.
Also added a short test to ensure urlopen is called with the timeout value.
Co-Authored-By: Alistair Coles <email address hidden>
Change-Id: Ic08a55157fa91f
Partial-Bug: 1419916
commit 376ecc74fccef39
Author: John Dickinson <email address hidden>
Date: Wed Apr 1 12:41:25 2015 -0700
make `git review` easier
Change-Id: I6032a7915586db
commit 8d3b3b2ee0a86b3
Author: Samuel Merritt <email address hidden>
Date: Sun Jan 11 13:41:35 2015 -0800
Add some debug output to the ring builder
Sometimes, I get handed a builder file in a support ticket and a
question of the form "why is the balance [not] doing $thing?". When
that happens, I add a bunch of print statements to my local
swift/
print statements. This time, instead of deleting the print statements,
I turned them into debug() calls and added a "--debug" flag to the
rebalance command in hopes that someone else will find it useful.
Change-Id: I697af90984fa5b
commit cf21db4bc68aff1
Author: Clay Gerrard <email address hidden>
Date: Thu Mar 26 21:43:48 2015 -0700
Add Swift Design Principles to CONTRIBUTING.md
This is based of work started a few design summits ago [1].
With EC getting close I felt like I could the inspiration.
1. https:/
Change-Id: I90fa56d6700308
Signed-off-by: Thiago da Silva <email address hidden>
commit a8129410030eb03
Author: Tushar Gohad <email address hidden>
Date: Fri Mar 13 01:36:53 2015 -0700
Bump eventlet version to 0.16.1
This version adds support for multiple 100-continue responses
via the API send_hundred_
the two-phase scheme for PUTs with Erasure Coding
Change-Id: I7154ef1dd96ab7
commit 75e84543aab5af8
Author: Hisashi Osanai <email address hidden>
Date: Mon Mar 23 11:29:16 2015 +0900
Clarify the description of backward compatibility in Keystoneauth's docstring
If no prefix implies for the operator_roles and/or service_roles
parameters applies to all reseller_prefixes in the code.
But from backward comatibility point of view, the service_roles
parameter is newly introduced parameter so it is better to clarify
the docstring.
Change-Id: I8b0d5aeff785c2
commit 76b106fc0179229
Author: Martin Kletzander <email address hidden>
Date: Thu Mar 19 17:25:57 2015 +0100
Fix common misspellings
Wikipedia's list of common misspellings [1] has a machine-readable
version. This patch fixes those misspellings mentioned in the list
which don't have multiple right variants (as e.g. "accension", which can
be both "accession" and "ascension"), such misspellings are left
untouched. The list of changes was manually re-checked for false
positives.
[1] https:/
Change-Id: Ic9a5438629664f
Signed-off-by: Martin Kletzander <email address hidden>
commit a288d05ca5d4c66
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Mar 24 06:06:33 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https:/
Change-Id: I6ae0dd0c34ac1d
commit 0a4679366210094
Author: Lorcan <email address hidden>
Date: Thu Mar 12 15:40:39 2015 +0000
Add swift-recon feature to track swift-drive-audit error count
This is a follow-on from a previous commit which added recon info
for swift-drive-audit (https:/
Here, the "--drievaudit" option is added to swift-recon tool. This
feature gives the statistics for the system-wide drive errors flagged
by swift-drive-audit. An example of the output is as follows:
(verbose mode)
swift-recon --driveaudit -v
===
--> Starting reconnaissance on 5 hosts
===
[2015-03-11 17:13:39] Checking drive-audit errors
-> http://
-> http://
-> http://
-> http://
-> http://
[drive_
===
Change-Id: Ia16c52a9d613ee
commit 23d0842dec25090
Author: Thiago da Silva <email address hidden>
Date: Tue Feb 17 16:55:34 2015 -0500
Refactoring the PUT method
Extracting large chunks of the PUT method into smaller
methods to improve maintainability and reuse of code.
Based on the work that Clay Gerrard started:
https:/
Co-Authored-By: Clay Gerrard <email address hidden>
Change-Id: Id479fc5b159a27
Signed-off-by: Thiago da Silva <email address hidden>
commit c3d2602c60d8a21
Author: Mitsuhiro SHIGEMATSU <email address hidden>
Date: Sat Mar 21 08:42:23 2015 +0900
Fix typo in swift/test/
Change-Id: I6cab1105990194
commit 7e64c811fe9ecae
Author: Alistair Coles <email address hidden>
Date: Wed Feb 25 17:33:44 2015 +0000
Enable in-process functional test policy to be configured
Currently the in-process tests build a 2 replica, 4 partition,
2 device object ring. This patch allows an alternative policy
and ring to be specified for testing via environment variables
that may optionally be set.
SWIFT_
directory which may have a swift.conf file and ring file. The
test setup will then prefer these conf files over the samples
in '/etc'.
SWIFT_
use the specified policy from the swift.conf file and its
associated ring for testing (first copying the conf and ring file
and modifying device parameters to suit in-process testing). If
not set, the tests will use the default policy.
The in-process tests now start sufficient object servers for the
ring file being tested against.
This should allow in-process functional testing of various policies
and rings (e.g. EC policies) without needing to reconfigure an SAIO
for each test scenario.
The refactoring of the in_process test setup code should also
allow easier addition of other 'hard-coded' test policies/rings
in the future.
Change-Id: I24f5a13de3d296
commit 9ae0a0702c4a324
Author: Kota Tsuyuzaki <email address hidden>
Date: Fri Mar 20 00:59:15 2015 -0700
Fix a lack of method arguments at tempauth
This fixes a kind of trivial issue with TypeError caused by a lack
of method argument at calling.
When using 'Authorization' header with tempauth, tempauth calls
_get_
but the method requires 4 arguments so it will go to fail with 500.
This patch fixes to take correct arguments and adds an unit test
for the case.
Closes-
Change-Id: Ibf00c41fa8e5cf
commit a624a593df7e92b
Author: Alistair Coles <email address hidden>
Date: Thu Mar 19 11:02:09 2015 +0000
Tighten up recon middleware unit tests
Adds a test for get_expirer_info() and tightens up
the test for get_async_info().
While reviewing [1] I noticed that the existing tests on
master will not catch a bug creeping into the arg strings
that these methods pass to from_recon_cache().
[1] https:/
Change-Id: Iee6f12aeff684d
commit 15b83f67d2fc664
Author: Alistair Coles <email address hidden>
Date: Wed Mar 18 12:43:03 2015 +0000
Test swift-object-info opens meta and ts files
Adds a unit test to verify the change made in [1], i.e. that
swift-
as .data files.
[1] change I43966d371218ad
Change-Id: I82dde36e3a96db
Related-Bug: 1425679
commit 3f1feaee4215dde
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Mar 18 06:11:23 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https:/
Change-Id: I2e0a6053fec208
commit 3d3db0ab789f37a
Author: Christian Schwede <email address hidden>
Date: Fri Mar 13 13:19:44 2015 +0000
Bump PyECLib version to >= 1.0.3
ECPyECLibEx
API/error handling enhancements in 1.0.3. Let's make sure
the right version is used.
Change-Id: Ie6e0c1b491017d
commit fdb2ec0cc742845
Author: Jiangmiao Gao <email address hidden>
Date: Mon Mar 16 23:30:37 2015 +0800
Add the missing "file" keyword for print
All non-keyword arguments are converted to strings and written to
the stream, the argument "sys.stderr" should be given keyword
"file" as keyword argument.
Change-Id: I1f9c4e41ed7a1d
commit 51a8f9c64fa16a2
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Mar 16 06:10:17 2015 +0000
Imported Translations from Transifex
For more information about this automatic import see:
https:/
Change-Id: I0058932520e008
commit 99a3d1a1c5d9591
Author: Kota Tsuyuzaki <email address hidden>
Date: Fri Mar 13 04:39:16 2015 -0700
Fix the prefix of messages caputured from stderr
Swift caputures both sys.stdout and sys.stderr messages and
transfers them to syslog. However, whole of the messages
uses the "STDOUT:" prefix, currently.
It will be quite confusable for debugging purpose because we
cannot make sure whether "Swift didn't caputure stderr" or
"Swift captured stderr but didn't show it" when we want to log
some stderr messages.
This patch enables to allow a new prefix "STDERR:" and use it
for sys.stderr.
Change-Id: Idf4649e9860b77
commit 9baafe7f305147c
Author: Ricardo Ferreira <email address hidden>
Date: Fri Mar 6 21:18:48 2015 +0000
Allow swift-object-info to inspect .meta and .ts files
Removing the check for ".data" extension makes it check
*any* file for metadata, so it works with .meta and .ts
filetypes.
Change-Id: I43966d371218ad
Closes-
commit 0eb27b96f267a1a
Author: Prashanth Pai <email address hidden>
Date: Thu Mar 5 19:01:38 2015 +0530
Handle ENOSPC in mkstemp()
mkstemp() can fail with ENOSPC when filesystem runs out of inodes.
And fallocate() used to raise DiskFileNoSpace for all OSErrors.
Change-Id: I8c95cb710107d8
Signed-off-by: Prashanth Pai <email address hidden>
commit 16b435a4a8ae12f
Author: Kota Tsuyuzaki <email address hidden>
Date: Mon Feb 23 00:26:30 2015 -0800
Fix ContainerBroker to use policy-0 in default
Fix ContainerBroker to initialize as policy-0 on policy_stat
table in default when storage_
Current ContainerBroker makes policy-1 stats in default because
the "None" value will be passed through to the last function of
db access query (i.e. a query like as "INSERT INTO policy_stat
(storage_
in a row "(1, 0, 0)" (the first value is the policy index) by
the PRIMARY KEY constraint on sqlite.
In worst case, container db keeps two policies, and then, ContainerBroker
might return invalid (non-touched) policy_stat information as container information.
(See tests in detail)
When using ContainerBroker with no storage_
it should always act policy-0 simply.
Note that this patch doesn't affect immediately Swift behavior because
current swift ensures to use policy-0 on "Container-Server" when invalid
policy (includes None) is given. However, we should recheck also in
ContainerBroker for safety to prevent the unfortunate behavior above.
Change-Id: If64f0c94c069a2
| Lana (loquacity) wrote | #1 |
| Changed in openstack-manuals: | |
| status: | New → Confirmed |
| importance: | Undecided → Medium |
| Changed in openstack-manuals: | |
| assignee: | nobody → janonymous (janonymous) |
| Changed in openstack-manuals: | |
| assignee: | janonymous (janonymous) → nobody |
| Changed in openstack-manuals: | |
| assignee: | nobody → Alexandra Settle (alexandra-settle) |
| Alexandra Settle (alexandra-settle) wrote | #2 |
| Changed in swift: | |
| status: | New → Confirmed |
| assignee: | nobody → Alexandra Settle (alexandra-settle) |
| Changed in openstack-manuals: | |
| status: | Confirmed → Incomplete |
| Changed in openstack-manuals: | |
| status: | Incomplete → Won't Fix |
| status: | Won't Fix → Incomplete |
| clayg (clay-gerrard) wrote | #3 |
| Changed in swift: | |
| importance: | Undecided → Medium |
| summary: |
- Fix best response to return correct status + Update Container-Sync Docs for internal_client.conf |
| Alexandra Settle (alexandra-settle) wrote | #4 |
| Changed in swift: | |
| assignee: | Alexandra Settle (alexandra-settle) → nobody |
| Changed in openstack-manuals: | |
| status: | Incomplete → Won't Fix |
| Changed in swift: | |
| assignee: | nobody → Kazuhiro MIYAHARA (miyahara-kazuhiro) |
| OpenStack Infra (hudson-openstack) wrote Fix merged to swift (master) | #5 |
| Changed in swift: | |
| status: | Confirmed → Fix Released |
| OpenStack Infra (hudson-openstack) wrote Fix included in openstack/swift 2.16.0 | #6 |
<quote>
Update container sync to use internal client
This patch changes container sync to use Internal Client instead
of Direct Client.
In the current design, container sync uses direct_get_object to
get the newest source object(which talks to storage node directly).
This works fine for replication storage policies however in
erasure coding policies, direct_get_object would only return part
of the object(it's encoded as several pieces). Using Internal
Client can get the original object in EC case.
Note that for the container sync put/delete part, it's working in
EC since it's using Simple Client.
Signed-off-by: Yuan Zhou <email address hidden>
DocImpact
</quote>
This needs to be updated in dev docs.