Comment 5 for bug 1844822

Revision history for this message
Nathaniel Sherry (nsherry4) wrote :

We were having the same issue on Stein:

Exception in thread privsep_reader:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 765, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/site-packages/oslo_privsep/comm.py", line 130, in _reader_main
for msg in reader:
File "/usr/lib/python2.7/site-packages/six.py", line 564, in next
return type(self).__next__(self)
File "/usr/lib/python2.7/site-packages/oslo_privsep/comm.py", line 77, in __next__
return next(self.unpacker)
File "msgpack/_unpacker.pyx", line 562, in msgpack._cmsgpack.Unpacker.__next__
File "msgpack/_unpacker.pyx", line 493, in msgpack._cmsgpack.Unpacker._unpack
ValueError: 1068129 exceeds max_bin_len(1048576)

We were able to fix it by increasing the max_buffer_size of the msgpack.Unpacker created by oslo_privsep.comm.Deserializer from the default 1024*1024 to something larger:

class Deserializer(six.Iterator):
    def __init__(self, readsock):
        self.readsock = readsock
        self.unpacker = msgpack.Unpacker(use_list=False, encoding='utf-8',
                                         unicode_errors='surrogateescape',
                                         max_buffer_size=1024*1024*8)