Remove broken translations from replicator logs

[Andrey Groshrev]

Hi, All!
After update swift to "newton" I saw errors in logs.

object-replicator: Ошибка синхронизации с узлом {'index': 0, u'replication_port': 60001, u'weight': 100.0, u'zone': 4, u'ip': u'192.168.1.4', u'region': 1, u'id': 21, u'replication_ip': u'192.168.1.4', u'meta': u'', u'device': u's2data01', u'port': 60001}: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/obj/replicator.py", line 467, in update#012 remote_hash.get(suffix, -1)]#012 File "/usr/lib/python2.7/site-packages/swift/obj/replicator.py", line 157, in sync#012 return self.sync_method(node, job, suffixes, *args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/swift/obj/replicator.py", line 248, in rsync#012 args.append(join(rsync_module, node['device'],#012 File "/usr/lib/python2.7/site-packages/swift/obj/replicator.py", line 208, in _rsync#012 log_method(#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1430, in info#012 self.logger.info(msg, *args, **kwargs)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1149, in info#012 self._log(INFO, msg, args, **kwargs)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1268, in _log#012 self.handle(record)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1278, in handle#012 self.callHandlers(record)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1318, in callHandlers#012 hdlr.handle(record)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 749, in handle#012 self.emit(record)#012 File "/usr/lib64/python2.7/logging/handlers.py", line 843, in emit#012 msg = self.format(record) + '\000'#012 File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format#012 return fmt.format(record)#012 File "/usr/lib/python2.7/site-packages/swift/common/utils.py", line 1747, in format#012 record.message = record.getMessage()#012 File "/usr/lib64/python2.7/logging/__init__.py", line 328, in getMessage#012 msg = msg % self.args#012UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 8: ordinal not in range(128)

I tried to understand the essence of the problem:
Firstly, I did not expect that this server is used ru_RU.UTF-8 system locale. :)
The easiest way to remove the errors - change locale in system "localectl set-locale LANG=en_US.UTF-8".
But, Swift himself became localized and use national locale - is a normal.

In general, the problem in this particular case:
After rsync successfully done work, swift happily trying to tell me about it on my language.

As this:
log_method(_("Successful rsync of %(src)s at %(dst)s (%(time).03f)"),
                {'src': args[-2], 'dst': args[-1], 'time': total_time})

After translate through _ - log_method takes two arguments:
Успешное выполнение rsync для %(src)s на %(dst)s (%(time).03f)
{'src': u'/srv/swift2/s2data07/objects/238663/3dc', 'dst': u'192.168.1.4::swift2object/s2data05/objects/238663', 'time': 0.19800186157226562}

A first arg - normal string with non-ascii characters and second with unicode strings.
As default, python used "ascii" decoder and as result we got error.

I'm not sure how to right fix it, but I assume that there are two versions.

1. We assume that the default encoding is UTF-8 and change only swift/obj/replicator.py
log_method(_("Successful rsync of %(src)s at %(dst)s (%(time).03f)").decode('utf-8'),
                {'src': args[-2], 'dst': args[-1], 'time': total_time})
But, This problem may emerge in another place and not all exclusively use UTF-8.

2. Try get codec from gettext and change finction gettext_ in swift/__init__.py, as
def gettext_(msg):
    return _t.gettext(msg).decode(_t.charset())

Both options work for me, but I'm not a strong expert in python.

[clayg (clay-gerrard)]

I think there was some resolution OpenStack made that log messages shouldn't be translated - so probably this bug should be that.

[Mahati Chamarthy (mahati-chamarthy)]

https://review.openstack.org/#/c/446762/ -> patch that was approved to drop log message translation. (Thanks acoles for the link)

[Andrey Groshrev (greenx)]

The release Pike has already come out and the error has not been corrected. Does not anyone need anything?

proxy-server[11218]: Ошибка. Необрабатываемая исключительная ситуация в запросе: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/proxy/server.py", line 521, in handle_request#012 return handler(req)#012 File "/usr/lib/python2.7/site-packages/swift/proxy/controllers/base.py", line 283, in wrapped#012 return func(*a, **kw)#012 File "/usr/lib/python2.7/site-packages/swift/proxy/controllers/obj.py", line 741, in PUT#012 req, data_source, nodes, partition, outgoing_headers)#012 File "/usr/lib/python2.7/site-packages/swift/proxy/controllers/obj.py", line 930, in _store_object#012 req, nodes, partition, outgoing_headers, policy)#012 File "/usr/lib/python2.7/site-packages/swift/proxy/controllers/obj.py", line 576, in _get_put_connections#012 putters = [putter for putter in pile if putter]#012 File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 228, in next#012 return self.waiters.get().wait()#012 File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait#012 return self._exit_event.wait()#012 File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait#012 return hubs.get_hub().switch()#012 File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch#012 return self.greenlet.switch()#012 File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main#012 result = function(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/swift/proxy/controllers/obj.py", line 557, in _connect_put_node#012 _('Expect: 100-continue on %s') % req.swift_entity_path)#012 File "/usr/lib/python2.7/site-packages/swift/proxy/server.py", line 651, in exception_occurred#012 **kwargs)#012 File "/usr/lib/python2.7/site-packages/swift/common/utils.py", line 1759, in exception#012 call('%s: %s' % (msg, emsg), *args, **kwargs)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1446, in error#012 self.logger.error(msg, *args, **kwargs)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1175, in error#012 self._log(ERROR, msg, args, **kwargs)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1268, in _log#012 self.handle(record)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1278, in handle#012 self.callHandlers(record)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 1318, in callHandlers#012 hdlr.handle(record)#012 File "/usr/lib64/python2.7/logging/__init__.py", line 749, in handle#012 self.emit(record)#012 File "/usr/lib64/python2.7/logging/handlers.py", line 843, in emit#012 msg = self.format(record) + '\000'#012 File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format#012 return fmt.format(record)#012 File "/usr/lib/python2.7/site-packages/swift/common/utils.py", line 1817, in format#012 record.message = record.getMessage()#012 File "/usr/lib64/python2.7/logging/__init__.py", line 328, in getMessage#012 msg = msg % self.args#012UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in p...

[Andrey Groshrev (greenx)]

This is not an innocuous logging error - when an exception is thrown, the current pool queue is also killed.

[clayg (clay-gerrard)]

A welcome patch to fix this bug would find the string that's blowing up and get rid of the gettext wrapper:

i.e.

    - _('stuff we care about'), ...
    + 'stuff we care about', ...

[Andrey Groshrev (greenx)]

https://github.com/openstack/swift/pull/27

[Andrey Groshrev (greenx)]

or https://review.openstack.org/#/c/513049/

[Andrey Groshrev (greenx)]

Are you sure that we need to delete the translations?
The natural desire of man: to receive logs in his native language.
In the code a lot of places where a hard binding to the encoding UTF-8.

The proposed solution at the beginning will solve 99% of the problems.
Why does not it suit you?

Or We can come up with a parameter in swift.conf - "error language" or "default encoding".
Let's think again, why remove support for translations into national languages?