Crash when using DIGEST-MD5 with SSF>=128

[Andreas Hasenack]

I'm still troubleshooting this, but at the moment apps negotiating a DIGEST-MD5 authentication and requesting some form of transport encryption (ssf != 0) are crashing. The only example I have so far is the openldap client tools (so just one app really).

ssf=0 works:
$ ldapwhoami -U ubuntu@lxd -w ubuntusecret -O maxssf=0
SASL/DIGEST-MD5 authentication started
SASL username: ubuntu@lxd
SASL SSF: 0
dn:uid=ubuntu@lxd,cn=vms,cn=digest-md5,cn=auth

ssf=128 crashes:
$ ldapwhoami -U ubuntu@lxd -w ubuntusecret -O maxssf=128
SASL/DIGEST-MD5 authentication started
SASL username: ubuntu@lxd
SASL SSF: 128
SASL data security layer installed.
Segmentation fault (core dumped)

The crash seems to be inside openssl. I'll get a proper stack trace.

2.1.27, also built with openssl3, does not crash. So far only 2.1.28 (in kinetic-proposed).

[Andreas Hasenack (ahasenack)]

It's also crashing in debian: https://ci.debian.net/data/autopkgtest/unstable/amd64/p/python-bonsai/21842977/log.gz

[Andreas Hasenack (ahasenack)]

Working theory at the moment is that cyrus-sasl2 is using RC4 from OpenSSL, and OpenSSL3 deprecated it:

On Kinetic:
$ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)

$ echo -ne test | openssl rc4 -k test
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
Error setting cipher RC4
4057FE8C0B7F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:../crypto/evp/evp_fetch.c:349:Global default library context, Algorithm (RC4 : 37), Properties ()
Salted__gG

On Impish:
$ openssl version
OpenSSL 1.1.1l 24 Aug 2021

$ echo -ne test | openssl rc4 -k test
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
Salted__~T�|=�ʇ����

Jammy:
$ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
$ echo -ne "test" | openssl rc4 -k test
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
Error setting cipher RC4
40078BF4127F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:../crypto/evp/evp_fetch.c:349:Global default library context, Algorithm (RC4 : 37), Properties ()
Salted__��N�x���

Both jammy and kinetic show "Error setting cipher RC4".

Oh, and the stack trace confirming it's inside openssl:
(gdb) bt
#0 0x00007ffff74085cb in EVP_EncryptUpdate (ctx=0x0, out=out@entry=0x5555555c7cf4 "0\036\002\001\004w\031\200\027\061.3.6.1.4.1.4203.1.11.3ST-MD5 client step 3",
    outl=outl@entry=0x7fffffffdbc4, in=0x5555555c8d50 "0\036\002\001\004w\031\200\027\061.3.6.1.4.1.4203.1.11.311.311.3", inl=32) at ../crypto/evp/evp_enc.c:614
#1 0x00007ffff70a07a9 in enc_rc4 (text=0x555555585e00, input=<optimized out>, inputlen=<optimized out>, digest=0x7fffffffdc20 "^\316@+\322}\a\334\006T\005\353:H}\036\260l\\UUU",
    output=0x5555555c7cf4 "0\036\002\001\004w\031\200\027\061.3.6.1.4.1.4203.1.11.3ST-MD5 client step 3", outputlen=0x7fffffffdda4) at ../../plugins/digestmd5.c:1201
#2 0x00007ffff70a1ddb in digestmd5_encode (context=0x555555585e00, invec=<optimized out>, numiov=<optimized out>, output=0x55555559e708, outputlen=0x7fffffffdda4)
    at ../../plugins/digestmd5.c:1552
#3 0x00007ffff7f33c3e in _sasl_encodev (conn=conn@entry=0x555555586cf0, invec=invec@entry=0x7fffffffdd70, numiov=numiov@entry=1, p_num_packets=p_num_packets@entry=0x7fffffffdd0c,
    output=output@entry=0x55555559e708, outputlen=outputlen@entry=0x7fffffffdda4) at ../../lib/common.c:359
#4 0x00007ffff7f360a1 in sasl_encodev (conn=conn@entry=0x555555586cf0, invec=invec@entry=0x7fffffffdd70, numiov=numiov@entry=1, output=output@entry=0x5...

[Andreas Hasenack (ahasenack)]

https://github.com/cyrusimap/cyrus-sasl/pull/668

[Andreas Hasenack (ahasenack)]

https://github.com/cyrusimap/cyrus-sasl/pull/653

https://github.com/cyrusimap/cyrus-sasl/issues/665

[Andreas Hasenack (ahasenack)]

Submitted python-bonsai DEP8 fixes to Debian via https://salsa.debian.org/python-team/packages/python-bonsai/-/merge_requests/1

[Andreas Hasenack (ahasenack)]

Submitted the cyrus-sasl2 fix to Debian via https://salsa.debian.org/debian/cyrus-sasl2/-/merge_requests/11

[Andreas Hasenack (ahasenack)]

Bileto is green: https://bileto.ubuntu.com/#/ticket/4852

[Andreas Hasenack (ahasenack)]

A fixed cyrus-sasl2 is in kinetic-proposed.

[Andreas Hasenack (ahasenack)]

Fixed in https://launchpad.net/ubuntu/+source/cyrus-sasl2/2.1.28+dfsg-6