ssync_sender encoding issue
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
High
|
Unassigned |
Bug Description
SSYNC sender crashes with the following traceback:
Mar 31 10:05:24 localhost object-
Traceback (most recent call last):
File "/opt/swift-
self.updates()
File "/opt/swift-
self.
File "/opt/swift-
msg = '\r\n'.join(msg) + '\r\n\r\n'
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 28: ordinal not in range(128)
It happens when the object name contains non-ascii char, and the metadata contains a key in unicode string. eg: df.get_
metadata = [
('name', '/AUTH_
(u'
...
]
When hitting this situation, the reconstruction for the partition stops, creating dispersion or missing fragments.
Changed in swift: | |
importance: | Undecided → High |
I'm having trouble duplicating this bug for replicated objects.
In my tests the handling of the object name and metadata in the object server is being treated consistently as utf-8 encoded bytes - not unicode strings.
object-6010: STDOUT: 'Content-Type': 'application/ octet-stream' 04e9800998ecf84 27e' Meta-Mtime' : '1490985870.469097' Meta-\xe2\ x98\x83' : '\xe2\x98\x83'
object-6010: STDOUT: 'ETag': 'd41d8cd98f00b2
object-6010: STDOUT: 'X-Object-
object-6010: STDOUT: 'X-Object-
object-6010: STDOUT: 'X-Timestamp': '1490987405.84984'
On master, the EC reconstructor seems to blow up in an entirely different spot:
object-6030: STDERR: Traceback (most recent call last): lib/python2. 7/dist- packages/ eventlet/ hubs/hub. py", line 457, in fire_timers lib/python2. 7/dist- packages/ eventlet/ hubs/timer. py", line 58, in __call__ lib/python2. 7/dist- packages/ eventlet/ greenthread. py", line 214, in main swift/swift/ common/ utils.py" , line 2726, in _run_func .put(func( *args, **kwargs)) swift/swift/ obj/reconstruct or.py", line 231, in _get_response swift/swift/ obj/reconstruct or.py", line 90, in _full_path
object-6030: STDERR: File "/usr/local/
object-6030: STDERR: timer()
object-6030: STDERR: File "/usr/local/
object-6030: STDERR: cb(*args, **kw)
object-6030: STDERR: File "/usr/local/
object-6030: STDERR: result = function(*args, **kwargs)
object-6030: STDERR: File "/vagrant/
object-6030: STDERR: self._responses
object-6030: STDERR: File "/vagrant/
object-6030: STDERR: full_path = _full_path(node, part, path, policy)
object-6030: STDERR: File "/vagrant/
object-6030: STDERR: 'policy': policy,
object-6030: STDERR: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 20: ordinal not in range(128)
I'm creating objects from the command line using u'\N{SNOWMAN}' in the name & metadata
swift upload test ☃ -H 'x-object-meta-☃: ☃'