I was trying to backport gnocchi to the trusty-mitaka ubuntu cloud archive (which builds the package on i386 only). Tests are executed as part of the package build. After several tests have run successfully (~1300), the remaining 222 tests fail with the the same "thread.error: can't start new thread" failure. For example:
======================================================================
FAIL: gnocchi.tests.test_rest.ResourceTest.test_post_resource_with_invalid_metric(keystone,image_none_user)
gnocchi.tests.test_rest.ResourceTest.test_post_resource_with_invalid_metric(keystone,image_none_user)
----------------------------------------------------------------------
_StringException: Empty attachments:
stderr
stdout
Traceback (most recent call last):
File "gnocchi/tests/base.py", line 53, in skip_if_not_implemented
return func(*args, **kwargs)
File "gnocchi/tests/test_rest.py", line 766, in setUp
super(ResourceTest, self).setUp()
File "gnocchi/tests/base.py", line 53, in skip_if_not_implemented
return func(*args, **kwargs)
File "gnocchi/tests/test_rest.py", line 102, in setUp
super(RestTest, self).setUp()
File "gnocchi/tests/base.py", line 53, in skip_if_not_implemented
return func(*args, **kwargs)
File "gnocchi/tests/base.py", line 420, in setUp
self.storage = storage.get_driver(self.conf)
File "gnocchi/storage/__init__.py", line 155, in get_driver
return get_driver_class(conf)(conf.storage)
File "gnocchi/storage/file.py", line 44, in __init__
super(FileStorage, self).__init__(conf)
File "gnocchi/storage/_carbonara.py", line 69, in __init__
self.heartbeater.start()
File "/usr/lib/python2.7/threading.py", line 745, in start
_start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread
Carbonara appears to be running out of threads. trusty python packages build on i386 only whereas xenial are on amd64, so that may be why this only surfaces on i386 for us.
Makes sense, the heartbeater thread is never stopped. We need to implement a stop() can call it.