Internal error in memory allocation

Bug #292604 reported by Timothy Allen on 2008-11-02
6
Affects Status Importance Assigned to Milestone
gnutls13 (Ubuntu)
High
Unassigned
Hardy
Undecided
Unassigned
gnutls26 (Debian)
Fix Released
Unknown

Bug Description

Upon upgrading an Exim mail server from Hardy to Intrepid (libgnutls26), mail from an Exim client running Hardy (libgnutls13) fails with a (client) error:

TLS error on connection to mail [xx.xx.xx.xx] (gnutls_handshake): Internal error in memory allocation.

This problem only occurs between a Hardy client and an Intrepid server, as far as I have been able to determine.

As far as I can tell, this is caused by the fix for debbug#478191, which is incorporated into libgnutls26.

Running gnutls-cli by hand yields this error and trace (attached).

Timothy Allen (tim-treehouse) wrote :
description: updated
Kjell Braden (afflux) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please try to obtain a backtrace following the instructions at http://wiki.ubuntu.com/DebuggingProgramCrash and upload the backtrace (as an attachment) to the bug report. This will greatly help us in tracking down your problem.

Changed in gnutls13:
status: New → Incomplete
Timothy Allen (tim-treehouse) wrote :

Thanks. Please find backtrace attached. (This is the first backtrace I've done for Ubuntu; please let me know if anything is incorrect.)

Timothy Allen (tim-treehouse) wrote :

As a follow-up, changing the line:
#define MAX_HANDSHAKE_PACKET_SIZE 16*1024
to
#define MAX_HANDSHAKE_PACKET_SIZE 48*1024
in lib/gnutls_int.h does resolve the problem, as suggested in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=478191

This seems to indicate that the fix should be applied to libgnutls13 in order for it to work with the already-updated libgnutls26.

Kees Cook (kees) wrote :

I saw this as well after doing a Hardy to Intrepid upgrade.

Changed in gnutls13:
importance: Undecided → High
status: Incomplete → Confirmed
Kees Cook (kees) wrote :

Hardy back-port of the handshake size option uploaded to -proposed.

Changed in gnutls13:
status: Confirmed → In Progress
Kees Cook (kees) wrote :
Kees Cook (kees) wrote :

A little more information about this -- as hinted at by the Debian bug: the cause of the problem on the server end is the giant number of certs being presented by the server to the client. I solved this temporarily by removing nearly all the certs from ca-certificates (sudo dpkg-reconfigure ca-certificates / ask / unselect many many / ok)

Martin Pitt (pitti) wrote :

This is fixed in intrepid and above.

Changed in gnutls13:
status: New → Fix Committed
status: In Progress → Fix Released
Martin Pitt (pitti) wrote :

Accepted into hardy-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Martin Pitt (pitti) wrote :

Any testers?

Jamie Strandboge (jdstrand) wrote :

This will be superceded by 2.0.4-1ubuntu2.5 very soon. 2.0.4-1ubuntu2.5 fixes a high priority regression (bug #305264).

Steve Beattie (sbeattie) wrote :

It looks like the fix for this did indeed get dropped in the superceded 2.0.4-1ubuntu2.5 package, and no one has merged back in this fix. I added a testcase for it to the lp:qa-regression-testing bzr tree.

Changed in gnutls13 (Ubuntu Hardy):
status: Fix Committed → In Progress
tags: removed: verification-needed
Martin Pitt (pitti) wrote :

Since nobody showed interest in testing this, I set this to wontfix for now. If this affects you, and you are able and willing to test, please shout here, then we can re-merge the fix.

Changed in gnutls13 (Ubuntu Hardy):
status: In Progress → Won't Fix
Timothy Allen (tim-treehouse) wrote :

I can't currently replicate the issue between a hardy client and a jaunty server (the original problem occurred when connecting to an intrepid server). As such, I would agree that the problem appears to be resolved.

Steve Beattie (sbeattie) wrote :

I believe it's still an open issue, based on the test I wrote at http://bazaar.launchpad.net/~ubuntu-bugcontrol/qa-regression-testing/master/revision/492?remember=490&compare_revid=490 . Basically, I (think I) can reproduce it by setting up a test server with the attached ca-certificates.crt file from an intrepid installation like so:

  gnutls-serv -p 4433 --x509keyfile /etc/ssl/private/ssl-cert-snakeoil.key --x509certfile /etc/ssl/certs/ssl-cert-snakeoil.pem --x509cafile lp292604-ca-certificate.crt

and then connecting to it with the gnutls client via:

  gnutls-cli -V 4433 --insecure [server]

This succeeded with the following gnutls clients:

  * intrepid
  * jaunty
  * gnutls-cli/libgnutls13 from hardy/2.0.4-1ubuntu2.4 (manually downloaded from the builds at https://launchpad.net/ubuntu/+source/gnutls13/2.0.4-1ubuntu2.4)

It failed with gnutls clients/libs from:

  * hardy/2.0.4-1ubuntu2
  * hardy-security/2.0.4-1ubuntu2.3
  * hardy-proposed/2.0.4-1ubuntu2.5

(In all of these, it didn't matter which release the gnutls-serv was from, the important bit was the ca-certificates.crt file.)

Also, if I removed the "--x509cafile lp292604-ca-certificate.crt" argument, all versions worked, even the ones that failed before. If I used the smaller ca-certificate.crt from a hardy installation, all versions succeeded as well.

Based on all of the above, I believe this is still an issue in 2.0.4-1ubuntu2.5, the current version in hardy-proposed.

Jamie Strandboge (jdstrand) wrote :

gnutls13 (hardy) now has 2.0.4-1ubuntu2.5 and seems to still be affected by this bug.

Jon Higgs (jhiggs0) wrote :

libgnutls13 2.0.4-1ubuntu2.6 (hardy) is also still effected.

Changed in gnutls26 (Debian):
status: Unknown → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.