bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh

Bug #578226 reported by Andrew Bennetts
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
Confirmed
Low
Unassigned

Bug Description

Failure found by babune: <http://babune.ladeuil.net:24842/job/selftest-lucid/lastCompletedBuild/testReport/bzrlib.tests.test_transport/TestSSHConnections/test_bzr_connect_to_bzr_ssh/>

The exception is:
SocketConnectionError: Unable to connect to SSH host localhost:39930; [Errno 111] Connection refused

See the link for the full traceback.

Reading the code I don't see any likely explanation. It *seems* that could only happen if the SocketListener has been started and then stopped before the connection happens?

Revision history for this message
Andrew Bennetts (spiv) wrote :

I've landed a little bit more logging in stub_sftp, so hopefully the next occurrence of this failure will provide some more hints.

Revision history for this message
Martin Pool (mbp) wrote :
Download full text (3.2 KiB)

I just hit this running selftest parallel=fork:

ERROR: bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh
----------------------------------------------------------------------
_StringException: Text attachment: log
------------
359.965 failed to load system host keys: [Errno 2] No such file or directory: '/tmp/testbzr-sIvONc.tmp/bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh/home/.ssh/known_hosts'
359.965 failed to load bzr host keys: [Errno 2] No such file or directory: '/tmp/testbzr-sIvONc.tmp/bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh/home/.bazaar/ssh_host_keys'
359.965 creating config directory: '/tmp/testbzr-sIvONc.tmp/bzrlib.tests.test_transport.TestSSHConnections.test_bzr_connect_to_bzr_ssh/home/.bazaar'
359.970 opening working tree '/tmp/testbzr-sIvONc.tmp'
------------
Text attachment: traceback
------------
Traceback (most recent call last):
  File "/home/mbp/lib/python/testtools/runtest.py", line 144, in _run_user
    return fn(*args)
  File "/home/mbp/lib/python/testtools/testcase.py", line 442, in _run_test_method
    testMethod()
  File "/home/mbp/bzr/work/bzrlib/tests/test_transport.py", line 981, in test_bzr_connect_to_bzr_ssh
    t.mkdir('foo')
  File "/home/mbp/bzr/work/bzrlib/transport/remote.py", line 242, in mkdir
    self._serialise_optional_mode(mode))
  File "/home/mbp/bzr/work/bzrlib/transport/remote.py", line 182, in _call2
    return self._client.call(method, *args)
  File "/home/mbp/bzr/work/bzrlib/smart/client.py", line 132, in call
    result, protocol = self.call_expecting_body(method, *args)
  File "/home/mbp/bzr/work/bzrlib/smart/client.py", line 145, in call_expecting_body
    method, args, expect_response_body=True)
  File "/home/mbp/bzr/work/bzrlib/smart/client.py", line 90, in _call_and_read_response
    readv_body=readv_body, body_stream=body_stream)
  File "/home/mbp/bzr/work/bzrlib/smart/client.py", line 63, in _send_request
    encoder.call(method, *args)
  File "/home/mbp/bzr/work/bzrlib/smart/protocol.py", line 1309, in call
    self._write_end()
  File "/home/mbp/bzr/work/bzrlib/smart/protocol.py", line 1133, in _write_end
    self.flush()
  File "/home/mbp/bzr/work/bzrlib/smart/protocol.py", line 1099, in flush
    self._real_write_func(''.join(self._buf))
  File "/home/mbp/bzr/work/bzrlib/smart/medium.py", line 396, in accept_bytes
    self._accept_bytes(bytes)
  File "/home/mbp/bzr/work/bzrlib/smart/medium.py", line 978, in _accept_bytes
    self._medium._accept_bytes(bytes)
  File "/home/mbp/bzr/work/bzrlib/smart/medium.py", line 794, in _accept_bytes
    self._ensure_connection()
  File "/home/mbp/bzr/work/bzrlib/smart/medium.py", line 818, in _ensure_connection
    '--directory=/', '--allow-writes'])
  File "/home/mbp/bzr/work/bzrlib/transport/ssh.py", line 332, in connect_ssh
    t = self._connect(username, password, host, port)
  File "/home/mbp/bzr/work/bzrlib/transport/ssh.py", line 287, in _connect
    self._raise_connection_error(host, port=port, orig_error=e)
  File "/home/mbp/bzr/work/bzrlib/transport/ssh.py", line 257, in _raise_connection_error
    orig_error=orig_error)
SocketConnectionError: ...

Read more...

tags: added: babune selftest ssh
Revision history for this message
Martin Pool (mbp) wrote :

Could it be that we're just not synchronizing with the server before trying to connect to it, therefore there's a timing and load dependent case where it has not yet listened on the socket?

Revision history for this message
Vincent Ladeuil (vila) wrote :

Jam's last fix was precisely about this point, so this shouldn't be the case anymore.

How many times should we hit issues with this test before we get rid of it ?

Revision history for this message
Vincent Ladeuil (vila) wrote :

Sorry, this sounds far more rude than intended, ctrl-atl-del:

I think this test has failed far too may times for wrong reasons so we should fix it by either:
- adding more tests for specific aspects that will fail and be easier to fix than this one,
- since it is already known to be slow, let spawn the server in a subprocess avoiding some intra-process issues,
- add a babune specific run via test-local-server to cover even more cases regarding a true ssh connection to a smart server

Jelmer Vernooij (jelmer)
tags: added: check-for-breezy
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.