selftest: thread.error: can't start new threadsts.test_put_file_non_atomic(RemoteTCPTransport,SmartTCPServer_for_testing)

[Christian Faulhammer]

I am not sure if this is related to 392127, so I report it anyway.

Running bzr selftest manually succeeds, but running the test suite from an ebuild yields the following traceback:

bzr: ERROR: thread.error: can't start new
threadsts.test_put_file_non_atomic(RemoteTCPTransport,SmartTCPServer_for_testing)

Traceback (most recent call last):
  File
"/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py",
line 141, in serve
    self.serve_conn(conn, thread_name_suffix)
  File
"/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py",
line 174, in serve_conn
    connection_thread.start()
  File "/usr/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

bzr 2.1.0 on python 2.6.4
(Linux-2.6.31-gentoo-r6-i686-Intel-R-_Core-TM-2_Duo_CPU_T8100_@_2.10GHz-with-gentoo-2.0.1)
arguments: ['bzr', '--no-plugins', 'selftest', '-x',
'(bzrlib.tests.blackbox.test_version.*|test_http.*)']
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: 'de_DE.UTF8'
plugins:

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
Exception in thread server-bzr://127.0.0.1:47705/extra/:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.6/threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File
"/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py",
line 141, in serve
    self.serve_conn(conn, thread_name_suffix)
  File
"/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py",
line 174, in serve_conn
    connection_thread.start()
  File "/usr/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

Traceback (most recent call
last):er_transport.TransportTests.test_put_file_non_atomic(SFTPTransport,SFTPHomeDirServer)
  File
"/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/tests/stub_sftp.py",
line 297, in run
    threading.Thread(target=self._callback, args=(s,)).start()
  File "/usr/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread
Exception in thread Thread-2994:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File
"/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/tests/stub_sftp.py",
line 306, in run
    warning('Exception from within unit test server thread: %r' %
NameError: global name 'warning' is not defined

[Vincent Ladeuil (vila)]

It's indeed the same root cause as bug #392127 but the final error is different.
I'll fix this one as well as bug #526221 since they both are fallouts from a recent refactoring.

The "can't create new thread" one should still be #392127

[Christian Faulhammer (fauli)]

Thank you so much, your patch works. But now I get another error. New bug?

bzr: ERROR: thread.error: can't start new threadmartServerAutopack.test_autopack_or_streaming_rpc_is_used_when_using_hpss(1.9)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py", line 141, in serve
    self.serve_conn(conn, thread_name_suffix)
  File "/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py", line 174, in serve_conn
    connection_thread.start()
  File "/usr/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

bzr 2.1.0 on python 2.6.4 (Linux-2.6.31-gentoo-r10-i686-Intel-R-_Core-TM-2_Duo_CPU_T8100_@_2.10GHz-with-gentoo-2.0.1)
arguments: ['bzr', '--no-plugins', 'selftest', '-x', '(bzrlib.tests.blackbox.test_version.*|test_http.*)']
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: 'de_DE.UTF8'
plugins:

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
Exception in thread server-bzr://127.0.0.1:43883/extra/:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.6/threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py", line 141, in serve
    self.serve_conn(conn, thread_name_suffix)
  File "/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py", line 174, in serve_conn
    connection_thread.start()
  File "/usr/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

bzr: ERROR: thread.error: can't start new threadtory.TestSmartServerAutopack.test_autopack_or_streaming_rpc_is_used_when_using_hpss(1.9-rich-

Traceback (most recent call last):
  File "/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py", line 141, in serve
    self.serve_conn(conn, thread_name_suffix)
  File "/var/tmp/portage/dev-util/bzr-2.1.0/work/bzr-2.1.0/bzrlib/smart/server.py", line 174, in serve_conn
    connection_thread.start()
  File "/usr/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

bzr 2.1.0 on python 2.6.4 (Linux-2.6.31-gentoo-r10-i686-Intel-R-_Core-TM-2_Duo_CPU_T8100_@_2.10GHz-with-gentoo-2.0.1)
arguments: ['bzr', '--no-plugins', 'selftest', '-x', '(bzrlib.tests.blackbox.test_version.*|test_http.*)']
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: 'de_DE.UTF8'
plugins:

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
Exception in thread server-bz...

[Vincent Ladeuil (vila)]

So, this failure is related to the thread/socket leaks in the tests (i.e. purely a test infrastructure problem,
bzr itself is not concerned).

As mentioned in bug #392127, a workaround is to run with --parallel=fork which
allows the test suite to complete without any errors (I'm running it this way every day in a VM with 4 cores).

If you don't have a multi-core processor, you can use BZR_CONCURRENCY=n to force it.

Add your new comments to bug #392127 or bug #392402, thanks in advance.

[Vincent Ladeuil (vila)]

This should be fixed for good in 2.3