subversion and GnuTLS internal error

[topaz]

Dear all,
I'm experiencing an interesting bug around gnutls.

My setup:
- svn server running apache2+mod_ssl under 12.04 precise (last upgrades done),
- svn client running 12.04 precise and providing a client ssl certificate,

The problem:
upon any svn operation (e.g. svn update), subversion output is:
svn: OPTIONS of 'https://server/svn/trunk': SSL handshake failed: SSL error: GnuTLS internal error. (https://server)

however if I use eclipse as my svn client, it works fine (with the same client certificate)

Workaround:
- reinstall (and force all) the package libgnutls26 version 2.10.5-1ubuntu3.1 from 11.10 Oneiric solves the problem but causes other applications to errors due to the broken libgcrypt11 dependency.

Has anyone experienced the same problem ?

Thanks in advance,
Cheers
Topaz.

[HS (hsass)]

I am seeing the same issue...

[topaz (top4z)]

I hope we'll have some feedback from gnutls team ...

[topaz (top4z)]

Just an update.

I've been experiencing troubles with my workaround, and had to edit manually my /var/lib/dpkg/status to ensure the previous version is accepted stealthy.

[...]
Package: libgnutls26
[...]
Version: 2.12.7-3
[...]

Tha drawback is that this workarount completely collaspe the version management.

Hope anyone in the gnu tls team will find out.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christopher Burke (krslynx)]

Hi,

I'm also facing this problem. Which steps did you take to recover?

Thanks

[topaz (top4z)]

To circumvent the problem :
1) I recompiled libneon WITHOUT gnutls, but using openssl instead (you need to desactivate sslv1, as the openssl version does not compile if you don't).
2) I used checkinstall to create a deb package for the recompiled lib
3) I changed the link of libneon27-gnutls to the version recompiled (still mentionning gnutls, but it's just to avoid breaking dependencies):
3.1) rm /usr/lib/libneon-gnutls.so.27
3.2) ln -s /usr/local/lib/libneon.so.27 /usr/lib/libneon-gnutls.so.27
4) I marked libneon27-gnutls "hold" to ensure not upgraded (using shell command #echo "libneon27-gnutls hold" | dpkg --set-selections)
5) restarted all kdesvn apps to ensure taking into account the new lib
6) I wrote the bug report.

I'm glad I'm not alone facing this damn annoying problem.

Topaz.

[Christopher Burke (krslynx)]

Hi,

I followed the above steps, and while it allows me to check out on the server locally (i.e. still using https://server.tld/svn/repo) I am still getting the error when checking out from remote machines.

Apologies in advance if these are newb questions, but how did you recompile libneon without gnutls? Did you grab the .deb or use some other means?

Thanks for the help!

Chris

[topaz (top4z)]

Hi Christopher,

Ok, it works for me, but I forgot to mention that the most important step is definitely the libneon without gnutls.
As the bug seems to be linked with gnutls, not evicting it means not circumventing the bug.

So I basically downloaded libneon sources and compiled them by hand. The easiest way is to use apt-get source:

here is the list of operation to recompile libneon27 without gnutls:

apt-get install build-essentials checkinstall
apt-get build-dep neon27
apt-get source neon27
cd /var/lib/neon27-0.29.6
./configure --with-ssl=openssl --prefix=/usr/local --enable-shared
fakeroot checkinstall --stripso=yes
dpkg -i libneon27-0.29.6....deb

# now replace the gnutls lib with the one compiled (to be redone upon upgrades)
rm /usr/lib/libneon-gnutls.so.27
ln -s /usr/local/lib/libneon.so.27 /usr/lib/libneon-gnutls.so.27

Enjoy :)
Topaz

[Christopher Burke (krslynx)]

Thanks for the extra help! I have done all of the above, however still getting the error. While trying to debug why I noticed this (amongst similar messages) in my access log:

5.96.78.5 - - [20/Aug/2012:13:10:55 +0100] "OPTIONS /fin HTTP/1.1" 200 3168 "-" "SVN/1.6.17 (r1128011) neon/0.29.3"

However, the version of Neon I just symlinked etc was neon/0.29.6

I updated my locate database, and searched for neon27-0.29.3 and couldn't find anything. Perhaps I need to re-compile apache or something?

Thanks,

Christopher

[Christopher Burke (krslynx)]

5.96.78.5 - /<email address hidden> [21/Aug/2012:17:17:46 +0100] "REPORT /svn/fin/!svn/vcc/default HTTP/1.1" 200 914 "-" "SVN/1.6.17 (r1128011) neon/0.29.6"

After going through the steps again, it looks like it's using the correct version of Neon now. However, I can still only check out from localhost rather than remotely.

[John Ryan (johnryannz)]

Hi all,

I am also affected by this bug exactly as described by Topaz (although I don't use eclipse so I don't have a working alternative at the moment). A while back there was a similar problem with client certificates and svn (see https://bugs.launchpad.net/ubuntu/+source/subversion/+bug/294648) for which the work-around was to install libneon27 and to set LD_PRELOAD=/usr/lib/libneon.so.27 before running svn. This worked at the time but now this produces a different error:

svn: OPTIONS of 'https://server/svn/trunk': SSL handshake failed: SSL disabled due to library version mismatch (https://server)

At present I have no way to use svn without disabling client certificates which is not really an option.

I haven't tried recompiling libneon yet but if Christopher is right and it will only fix checking out from localhost then it won't help me.

I am going to try a few debug builds of libneon27-gnutls and see if I can see what the problem is.

Cheers,
John

[John Ryan (johnryannz)]

Hi again,

I can add some new information now. As described by Topaz, if both the server and the client are running Precise then the problem is observed as described. If Oneiric is used as the client then the problem disappears suggesting that the issue is entirely client-side. Further, the version of libneon27-gnutls used by Precise and Oneiric is identical suggesting that the neon-gnutls library is not the problem (corroborated by the fact that installing libneon27 and using the LD_PRELOAD trick also doesn't work in Precise). However, the version of libgnutls26 between Oneiric and Precise is different and using LD_PRELOAD to load the Oneiric version of this library solves the problem in Precise.

Strangely however, Quantal also suffers from this problem but it can be fixed by using LD_PRELOAD to load the libneon27 library. LD_PRELOADing the libgnutls26 version from Oneiric doesn't work because libneon27-gnutls has a dependency on version 2.12 of libgnutls (Oneiric uses 2.10)!

Here is a summary:

"Out of the box": Oneiric OK, Precise fails, Quantal fails.

Install libneon27 and set LD_PRELOAD=/usr/lib/libneon.so.27: Oneiric OK, Precise fails, Quantal OK

Set LD_PRELOAD=libgnutls.so.26.16.14 (Oneiric version): Oneric OK, Precise OK, Quantal fails

Workarounds:

Precise: Download the Oneiric version of libgnutls.so.26.16.14 and set LD_PRELOAD to point to this file.
Quantal: Install libneon27 and set LD_PRELOAD=/usr/bin/libneon.so.27

For the record, the releases have the following versions of the libraries:

Oneiric
libneon27-gnutls_0.29.6-1 (library version libneon-gnutls.so.27.2.6)
libgnutls26_2.10.5-1ubuntu3.2 (library version libgnutls.so.26.16.14)

Precise
libneon27-gnutls_0.29.6-1 (library version libneon-gnutls.so.27.2.6)
libgnutls26_2.12.14-5ubuntu3.1 (library version libgnutls.so.26.21.8)

Quantal
libneon27-gnutls_0.29.6-3 (library version libneon-gnutls.so.27.2.6)
libgnutls26_2.12.14-5ubuntu4 (library version libgnutls.so.26.21.8)

[John Ryan (johnryannz)]

Whoops, quick correction - the workaround for Quantal is of course to set LD_PRELOAD=/usr/lib/libneon.so.27 not LD_PRELOAD=/usr/bin/libneon.so.27

Workarounds:

Precise: Download the Oneiric version of libgnutls.so.26.16.14 and set LD_PRELOAD to point to this file.
Quantal: Install libneon27 and set LD_PRELOAD=/usr/lib/libneon.so.27

[Andrew Colin Kissa (topdog)]

I have submitted a fix (debdiff) for this, if you can build and test.

https://bugs.launchpad.net/ubuntu/+source/gnutls26/+bug/1095052

[John Ryan (johnryannz)]

Hi Andrew,

I believe the patch works. I was unable to use dpkg-buildpackage to build a new gnutls deb because a number of the gnutls tests failed (due to an issue with libgcrypt?!). However, I could manually configure and build a new gnutls library and I was able to successfully use my client certificates with subversion to check out a source tree off my server.

Do you know if this upstream fix has made been incorporated into Quantal or Raring? My experience with Quantal would suggest not but I haven't tried Raring yet.

Thanks for your efforts - I hope this patch gets accepted and the new version released to the repositories soon.

Happy New Year,
John