tests failure in bzr-1.10: Failure when receiving data from the peer

Bug #306264 reported by pva on 2008-12-08
2
Affects Status Importance Assigned to Milestone
Bazaar
Low
Vincent Ladeuil

Bug Description

During running tests in Gentoo Linux some tests fail very similarly:

ERROR: blackbox.test_push.TestPush.test_push_new_branch_stacked_uses_parent_public
    (56, 'Failure when receiving data from the peer')

ERROR: test_bzrdir.ChrootedTests.test_open_containing_from_transport
    (56, 'Failure when receiving data from the peer')

[snip]

ERROR: test_http.TestWallServer.test_http_get(pycurl,HTTP/1.1)
    (56, 'Failure when receiving data from the peer')

ERROR: test_http.TestWallServer.test_http_has(pycurl,HTTP/1.1)
    (56, 'Failure when receiving data from the peer')

That's net-misc/curl-7.19.2 and dev-python/pycurl-7.16.4. With newer pycurl (7.19.0) I have even more similar test failures (removed 'Failure when receiving data from the peer' line):

ERROR: blackbox.test_too_much.HTTPTests.test_branch
ERROR: branch_implementations.test_branch.ChrootedTests.test_open_containing(BzrBranchFormat5)
ERROR: branch_implementations.test_branch.ChrootedTests.test_open_containing(RemoteBranchFormat-v2)
ERROR: bzrdir_implementations.test_bzrdir.ChrootedBzrDirTests.test_find_repository_no_repository(BzrDirMetaFormat1)
ERROR: test_bzrdir.ChrootedTests.test_open_containing_from_transport
ERROR: test_http.SmartHTTPTunnellingTest.test_http_send_smart_request(pycurl,HTTP/1.0)
    dictionary changed size during iteration

ERROR: test_http.SmartHTTPTunnellingTest.test_open_bzrdir(urllib,HTTP/1.0)
    dictionary changed size during iteration

[snip]

ERROR: test_http.TestWallServer.test_http_get(pycurl,HTTP/1.0)
ERROR: test_http.TestWallServer.test_http_get(pycurl,HTTP/1.1)
ERROR: test_http.TestWallServer.test_http_has(pycurl,HTTP/1.1)

Could you suggest me what could cause this? Also is it possible to disable tests selectively without removing this functions from code?

Related branches

pva (pva) wrote :
Vincent Ladeuil (vila) wrote :

This is most likely caused by an incompatible change in curl-7.19.1, http://curl.haxx.se/changes.html mentions:
Fixed in 7.19.1 - November 5 2008 :
<snip>
 Bugfixes:
<snip>
    * recv() failures cause CURLE_RECV_ERROR

56 is CURLE_RECV_ERROR

It's a bit strange that pycurl-7.16.4 cause *more* failures instead of less (obviously *something* has changed in pycurl regarding that error handling... or should have change).

Regarding running some tests only (if you want to do a better diagnosis) you can do:

bzr selftest -s bt.test_http.TestWallServer.test_http_get -s bt.test_http.TestWallServer.test_http_has etc

to run only the tests that errors out (selftest doesn't display the leading bzrlib.tests that is abbreviated 'bt' above)

I don't have [py]curl-7.19 installed so I can't reproduce myself.

Vincent Ladeuil (vila) wrote :

Also, I don't think our http client implementations have changed in 1.10, dp you see the same errors with bzr-1.9 ?

pva (pva) wrote :

vila, it's reverse: pycurl of version 7.19.0 cause more problems then pycurl 7.16.4 (with the same curl version installed). And speaking about 1.9 (full output in attachment) it looks like same tests fail (even more, but I suppose some bugs are fixed and some tests were added):

ERROR: blackbox.test_check.ChrootedCheckTests.test_check_missing_repository
ERROR: blackbox.test_outside_wt.TestOutsideWT.test_url_log
ERROR: branch_implementations.test_branch.ChrootedTests.test_open_containing(BzrBranchFormat5)
ERROR: branch_implementations.test_branch.ChrootedTests.test_open_containing(RemoteBranchFormat-default)
ERROR: test_bzrdir.ChrootedTests.test_open_containing
ERROR: test_bzrdir.ChrootedTests.test_open_containing_from_transport
ERROR: test_bzrdir.ChrootedTests.test_open_containing_tree_branch_or_repository_empty
ERROR: test_http.TestWallServer.test_http_get(pycurl,HTTP/1.1)
ERROR: test_http.TestWallServer.test_http_has(pycurl,HTTP/1.1)

About disabling tests, I mean, I need selectively disable tests which are known to fail. This helps me to work with our arch teams since they'll avoid test which are known to fail. And well, that's said, I'll move back to older py/curl to see how tests pass there...

>>>>> "pva" writes:

    pva> vila, it's reverse: pycurl of version 7.19.0 cause more problems then
    pva> pycurl 7.16.4 (with the same curl version installed).

I've seen that and that's what I find strange. It means that
there is a change in curl that leads to failures in bzr, one can
hope that a change in pycurl *fixes* some, not introduces more.

    pva> And speaking about 1.9 (full output in attachment) it
    pva> looks like same tests fail (even more, but I suppose
    pva> some bugs are fixed and some tests were added):

    pva> ERROR: blackbox.test_check.ChrootedCheckTests.test_check_missing_repository
    pva> ERROR: blackbox.test_outside_wt.TestOutsideWT.test_url_log
    pva> ERROR: branch_implementations.test_branch.ChrootedTests.test_open_containing(BzrBranchFormat5)
    pva> ERROR: branch_implementations.test_branch.ChrootedTests.test_open_containing(RemoteBranchFormat-default)
    pva> ERROR: test_bzrdir.ChrootedTests.test_open_containing
    pva> ERROR: test_bzrdir.ChrootedTests.test_open_containing_from_transport
    pva> ERROR: test_bzrdir.ChrootedTests.test_open_containing_tree_branch_or_repository_empty
    pva> ERROR: test_http.TestWallServer.test_http_get(pycurl,HTTP/1.1)
    pva> ERROR: test_http.TestWallServer.test_http_has(pycurl,HTTP/1.1)

I checked some of them and apparently they are tests which close
the connection early or send unsupported requests, i.e. error
situations anyway.

Probably curl is more strict now and returns an error code
instead of silently closing the connection.

    pva> About disabling tests, I mean, I need selectively disable tests which
    pva> are known to fail. This helps me to work with our arch teams since
    pva> they'll avoid test which are known to fail.

You can use '-x blackbox.test_check.ChrootedCheckTests.test_check_missing_repository' for that

    pva> And well, that's said, I'll move back to older py/curl
    pva> to see how tests pass there...

That would make our life easier, pycurl is only required these
days when ssl certificate verification is needed (which is worked
on but will require either python-2.6 or the ssl module for
python-2.5 or python-2.4).

Since this bug in particular is really about compatibility
between various packages versions inside a distribution I think
that was worth mentioning.

pva (pva) wrote :

I'm not sure I understood your comment about python and SSL module. Is it a separate package in python-2.5? I have python 2.5. with ssl USE flag installed. Do I need to install dev-python/pyopenssl or something like that? I've found that python 2.5 includes only basic ssl client functionality and for server side I need said module, but do I need that for bzr?

And yes, after downgrade, I have only one error which was reported elsewhere:
https://bugs.launchpad.net/bzr/+bug/277048

So confirmed this is problem with new py/curl libraries. I'll add a notice for Gentoo user about that until better fix will be found. Thank you for your time, vila.

Vincent Ladeuil (vila) wrote :

>>>>> "pva" writes:

    pva> I'm not sure I understood your comment about python and SSL module. Is
    pva> it a separate package in python-2.5?

Yes, sorry, I was unclear:

http://pypi.python.org/pypi/ssl/

    pva> I have python 2.5. with ssl USE flag installed.

That's good.
    pva> Do I need to install dev-python/pyopenssl or something
    pva> like that?

No.

    pva> I've found that python 2.5 includes only basic ssl
    pva> client functionality and for server side I need said
    pva> module,

Indeed. As you realized, that brings only a limited ssl support
which we use. That doesn't include certificate verification.

    pva> but do I need that for bzr?

Not yet, but the plan is to make it an optional but supported
module that will offer SSL certificate verification when present,
at which point we will be able to deprecate pycurl.

    pva> And yes, after downgrade, I have only one error which
    pva> was reported elsewhere:
    pva> https://bugs.launchpad.net/bzr/+bug/277048

Great. Err, great that there remains only one bug :)

    pva> So confirmed this is problem with new py/curl
    pva> libraries. I'll add a notice for Gentoo user about that
    pva> until better fix will be found.

Thanks.

Changed in bzr:
importance: Undecided → Low
status: New → Confirmed
Vincent Ladeuil (vila) wrote :

Finally, the same error (56, 'Failure when receiving data from the peer') occured on karmic !

I'll have a fix shortly.

Changed in bzr:
assignee: nobody → Vincent Ladeuil (vila)
Vincent Ladeuil (vila) on 2009-08-19
Changed in bzr:
status: Confirmed → Fix Committed
Vincent Ladeuil (vila) on 2009-09-08
Changed in bzr:
milestone: none → 2.0
milestone: 2.0 → 2.0rc1
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers