Comment 0 for bug 1002434

NSS (Netscape Security Services) module provides encryption services to many applications, such as Thunderbird, Firefox and Chromium. NSS has a hard coded maximum limit of 2236 bits for ephemeral Diffie-Hellman (DHE) keys. If the TLS server (such as a web server, SMTP server, IMAP server, etc) requests a bigger DHE key size, NSS based applications refuse to interoperate. They just close the connection and display a confusing error message (such as "Unknown error").

Recent versions of GnuTLS (as shipped by Ubuntu and other distributions) include a new library API which recommends and automatically selects the following key sizes:

Security level key bits

LOW 1248
LEGACY 1776
NORMAL 2432
HIGH 3248

See the following for more information: https://www.gnu.org/software/gnutls/manual/html_node/Selecting-cryptographic-key-sizes.html

As can be seen, NSS's maximum limit of 2236 bits can only interoperate with GnuTLS server which has been set at "LOW" or "LEGACY" security level.

This bug was discovered when Exim's GnuTLS interface was revamped recently. Thunderbird refused to complete TLS handshake with the Exim SMTP server any more, because the new GnuTLS interface was following the GnuTLS library's opinion on suitable key sizes.

Please patch the NSS library to accept reasonable key sizes: at the very least 3248 bits should be accepted to allow interoperability with GnuTLS at HIGH level. NSS is the only TLS library which has such a low hard limit on DHE key size.

The only reason people are not hitting this bug frequently yet is that most main stream server software still does not use GnuTLS library's new API or recommendations but instead hard codes the DHE key size to 1024 or 2048 bits.

I am attaching a patch which points out the relevant #define in blapit.h.