Internal error in memory allocation

[Timothy Allen]

Upon upgrading an Exim mail server from Hardy to Intrepid (libgnutls26), mail from an Exim client running Hardy (libgnutls13) fails with a (client) error:

TLS error on connection to mail [xx.xx.xx.xx] (gnutls_handshake): Internal error in memory allocation.

This problem only occurs between a Hardy client and an Intrepid server, as far as I have been able to determine.

As far as I can tell, this is caused by the fix for debbug#478191, which is incorporated into libgnutls26.

Running gnutls-cli by hand yields this error and trace (attached).

[Kjell Braden (afflux)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please try to obtain a backtrace following the instructions at http://wiki.ubuntu.com/DebuggingProgramCrash and upload the backtrace (as an attachment) to the bug report. This will greatly help us in tracking down your problem.

[Timothy Allen (tim-treehouse)]

Thanks. Please find backtrace attached. (This is the first backtrace I've done for Ubuntu; please let me know if anything is incorrect.)

[Timothy Allen (tim-treehouse)]

As a follow-up, changing the line:
#define MAX_HANDSHAKE_PACKET_SIZE 16*1024
to
#define MAX_HANDSHAKE_PACKET_SIZE 48*1024
in lib/gnutls_int.h does resolve the problem, as suggested in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=478191

This seems to indicate that the fix should be applied to libgnutls13 in order for it to work with the already-updated libgnutls26.

[Kees Cook (kees)]

I saw this as well after doing a Hardy to Intrepid upgrade.

[Kees Cook (kees)]

Hardy back-port of the handshake size option uploaded to -proposed.

[Kees Cook (kees)]

A little more information about this -- as hinted at by the Debian bug: the cause of the problem on the server end is the giant number of certs being presented by the server to the client. I solved this temporarily by removing nearly all the certs from ca-certificates (sudo dpkg-reconfigure ca-certificates / ask / unselect many many / ok)

[Martin Pitt (pitti)]

This is fixed in intrepid and above.

[Martin Pitt (pitti)]

Accepted into hardy-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

Any testers?

[Jamie Strandboge (jdstrand)]

This will be superceded by 2.0.4-1ubuntu2.5 very soon. 2.0.4-1ubuntu2.5 fixes a high priority regression (bug #305264).

[Steve Beattie (sbeattie)]

It looks like the fix for this did indeed get dropped in the superceded 2.0.4-1ubuntu2.5 package, and no one has merged back in this fix. I added a testcase for it to the lp:qa-regression-testing bzr tree.

[Martin Pitt (pitti)]

Since nobody showed interest in testing this, I set this to wontfix for now. If this affects you, and you are able and willing to test, please shout here, then we can re-merge the fix.

[Timothy Allen (tim-treehouse)]

I can't currently replicate the issue between a hardy client and a jaunty server (the original problem occurred when connecting to an intrepid server). As such, I would agree that the problem appears to be resolved.

[Steve Beattie (sbeattie)]

I believe it's still an open issue, based on the test I wrote at http://bazaar.launchpad.net/~ubuntu-bugcontrol/qa-regression-testing/master/revision/492?remember=490&compare_revid=490 . Basically, I (think I) can reproduce it by setting up a test server with the attached ca-certificates.crt file from an intrepid installation like so:

  gnutls-serv -p 4433 --x509keyfile /etc/ssl/private/ssl-cert-snakeoil.key --x509certfile /etc/ssl/certs/ssl-cert-snakeoil.pem --x509cafile lp292604-ca-certificate.crt

and then connecting to it with the gnutls client via:

  gnutls-cli -V 4433 --insecure [server]

This succeeded with the following gnutls clients:

  * intrepid
  * jaunty
  * gnutls-cli/libgnutls13 from hardy/2.0.4-1ubuntu2.4 (manually downloaded from the builds at https://launchpad.net/ubuntu/+source/gnutls13/2.0.4-1ubuntu2.4)

It failed with gnutls clients/libs from:

  * hardy/2.0.4-1ubuntu2
  * hardy-security/2.0.4-1ubuntu2.3
  * hardy-proposed/2.0.4-1ubuntu2.5

(In all of these, it didn't matter which release the gnutls-serv was from, the important bit was the ca-certificates.crt file.)

Also, if I removed the "--x509cafile lp292604-ca-certificate.crt" argument, all versions worked, even the ones that failed before. If I used the smaller ca-certificate.crt from a hardy installation, all versions succeeded as well.

Based on all of the above, I believe this is still an issue in 2.0.4-1ubuntu2.5, the current version in hardy-proposed.

[Jamie Strandboge (jdstrand)]

gnutls13 (hardy) now has 2.0.4-1ubuntu2.5 and seems to still be affected by this bug.

[Jon Higgs (jhiggs0)]

libgnutls13 2.0.4-1ubuntu2.6 (hardy) is also still effected.