Segfaults on verify callout, in _gnutls_trust_list_get_issuer
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
exim |
Fix Released
|
Unknown
|
|||
exim4 (Debian) |
Fix Released
|
Unknown
|
|||
exim4 (Ubuntu) |
Fix Released
|
High
|
Sergio Durigan Junior | ||
Jammy |
Fix Released
|
High
|
Sergio Durigan Junior | ||
Kinetic |
Fix Released
|
High
|
Sergio Durigan Junior |
Bug Description
[ Impact ]
When the user sends a message to someone, if the server responsible for receiving this message defers it, and if there are other possible servers (i.e., other servers listed as secondary MX) to try, exim4 will segfault while trying to connect to the second server.
[ Test Plan ]
The test case for this bug is a bit involved. It makes use of the upstream reporter's mail server, which has been configured to defer emails when they come through the primary MX, but accept when they come through the secondary MX. This means that you will need access to port 25 (unfortunately canonistack seems to block it).
$ lxc launch ubuntu-daily:jammy exim4-bug1974214
$ lxc shell exim4-bug1974214
# apt update && apt full-upgrade
# apt install -y exim4
# dpkg-reconfigure exim4-config
... In the "Mail Server configuration" screen, select "internet site; mail is sent and received directly using SMTP". Leave everything else untouched.
# cat > /etc/netplan/
network:
ethernets:
eth0:
link-local: [ ipv4 ]
_EOF_
# netplan apply
# reboot
$ lxc shell exim4-bug1974214
# cat > 1.msg << _EOF_
Subject: test
this is a test
_EOF_
# exim4 -odq -f <email address hidden> <email address hidden> < 1.msg
# exim4 -bp
0m 321 1nxC3o-0000Ax-AS <email address hidden>
<email address hidden>
... You will have to grab the message ID, which is 1nxC3o-0000Ax-AS in this case. You have to use this ID in the following command.
# exim4 -d+all -q 1nxC3o-0000Ax-AS 2>&1 | tee /tmp/exim.debug
...
# grep SEGV /tmp/exim.debug
You should be able to see exim4 signalling the segmentation fault that occurred while attempting to connect to the second server.
[ Where problems could occur ]
The patches, albeit well contained and relatively simple, touch code that deals with TLS and security. There is always the risk of introducing an unwanted vulnerability or normal regression here. If that happens, the very first thing we need to do is revert the patches and work with upstream to develop a fix.
In the unlikely case that we encounter regressions, they are probably going to affect TLS connections when sending/receiving messages. Email servers nowadays generally offer encrypted connections (via TLS or STARTTLS), and some still offer plaintext as well. If there is a problem with TLS and exim4 is configured to fallback to plaintext, things will still work assuming that the other end also talks plaintext. Otherwise, we might see reports of undelivered emails.
Finally, the fix is composed of two patches. The first one prevents exim4 from discarding the cached credentials when the transport connection with the primary MX closes, and the second resets headers before trying to connect to the secondary MX.
[ Original Description ]
We are experiencing segfaults in exim since upgrading from impish (4.94.2-7ubuntu2 with libgnutls30 3.7.1-5ubuntu1) to jammy (4.95-4ubuntu2 with libgnutls30 3.7.3-4ubuntu1), in _gnutls_
Typically the initial attempt to submit a message crashes an exim child thread, but the same message is accepted when the sender retries.
gdb backtrace:
Thread 2.1 "exim4" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fe2f844d080 (LWP 29278)]
0x00007fe2f8f3eb2b in _gnutls_
1026 x509/..
(gdb) bt
#0 0x00007fe2f8f3eb2b in _gnutls_
list=<optimised out>) at x509/..
#1 gnutls_
flags=
#2 0x00007fe2f8f3f679 in gnutls_
cert_
at x509/..
#3 0x00007fe2f8ed7516 in _gnutls_
at ../../lib/
#4 gnutls_
status=
#5 0x00007fe2f8ed8000 in gnutls_
at ../../lib/
#6 0x000055ef6b7698ef in verify_certificate (state=<optimised out>, errstr=
at /build/
#7 0x000055ef6b7a5d7b in tls_client_
tlsp=
at /build/
#8 0x000055ef6b78b0ef in smtp_setup_conn (sx=0x55ef6bdfe5e8, suppress_
#9 0x000055ef6b776350 in do_callout (pm_mailfrom=
callout_
host_
at /build/
#10 verify_address (vaddr=<optimised out>, fp=<optimised out>, options=<optimised out>, callout=<optimised out>,
callout_
routed=
#11 0x000055ef6b6f1660 in acl_verify (where=
arg=
log_
at /build/
#12 0x000055ef6b6f479e in acl_check_condition (level=<optimised out>, basic_errno=
user_
verb=<optimised out>) at /build/
#13 acl_check_internal (where=
user_
at /build/
#14 0x000055ef6b6f7b9e in acl_check (where=0, recipient=
user_
at /build/
#15 0x000055ef6b75c2fd in smtp_setup_msg () at /build/
#16 0x000055ef6b6e5cda in handle_smtp_call (accepted=
listen_
at /build/
#17 daemon_go () at /build/
#18 main (argc=<optimised out>, cargv=<optimised out>)
at /build/
Related branches
- git-ubuntu bot: Approve
- Bryce Harrington (community): Approve
- Canonical Server: Pending requested
-
Diff: 247 lines (+219/-0)4 files modifieddebian/changelog (+7/-0)
debian/patches/lp1974214-segfault-smtp-delivery-01.patch (+186/-0)
debian/patches/lp1974214-segfault-smtp-delivery-02.patch (+24/-0)
debian/patches/series (+2/-0)
- Bryce Harrington (community): Approve
- Canonical Server packageset reviewers: Pending requested
-
Diff: 247 lines (+219/-0)4 files modifieddebian/changelog (+7/-0)
debian/patches/lp1974214-segfault-smtp-delivery-01.patch (+186/-0)
debian/patches/lp1974214-segfault-smtp-delivery-02.patch (+24/-0)
debian/patches/series (+2/-0)
Changed in exim4 (Ubuntu): | |
status: | New → Confirmed |
summary: |
- Segfaults on sender verify callout, in _gnutls_trust_list_get_issuer + Segfaults on verify callout, in _gnutls_trust_list_get_issuer |
description: | updated |
Changed in exim4 (Ubuntu Kinetic): | |
assignee: | nobody → Sergio Durigan Junior (sergiodj) |
Changed in exim4 (Ubuntu Jammy): | |
assignee: | nobody → Sergio Durigan Junior (sergiodj) |
importance: | Medium → High |
Changed in exim4 (Ubuntu Kinetic): | |
importance: | Medium → High |
description: | updated |
tags: | added: server-todo |
Changed in exim4 (Ubuntu Jammy): | |
status: | Triaged → In Progress |
Changed in exim4 (Ubuntu Kinetic): | |
status: | Triaged → In Progress |
Changed in exim: | |
status: | Unknown → Fix Released |
Changed in exim4 (Debian): | |
status: | Unknown → Fix Released |
description: | updated |
description: | updated |
tags: |
added: verification-done removed: verification-needed |
tags: | removed: server-todo |
Created attachment 1414
exim4-daemon-light 4.96~RC0-1
Versions: 4.94 good, 4.9[56] bad.
OS: Debian testing x86_64
TLS: gnutls 3.7.4-2, haven't tried OpenSSL
This doesn't happen in an immediate delivery attempt or in "exim -M", but in a queue runner, if the first remote server responds with a deferral, exim crashes some time when talking to the next server. It can happen in tls_close() (after the message was successfully delivered, if the remote parry allows, or after getting a deferral): smtp_deliver -> tls_close -> gnutls_ certificate_ free_credential s -> gnutls_ x509_trust_ list_deinit,
or it can be: smtp_deliver -> smtp_setup_conn -> tls_client_start -> verify_certificate -> gnutls_ certificate_ verify_ peers2 -> gnutls_ certificate_ verify_ peers -> _gnutls_ x509_cert_ verify_ peers -> gnutls_ x509_trust_ list_verify_ crt2 -> gnutls_ x509_trust_ list_get_ issuer -> _gnutls_ trust_list_ get_issuer
And on one occasion it crashed in arc_sign() (on an exim thus built)
But long story short, it seems like exim would consistently crash, SIGFPE or SIGSEGV, during a subsequent delivery attempt after a deferral response.
The real life circumstances are a gmail account over quota or a forwarded message graylisted by gmail or such. But I am reproducing this by simply configuring my own exim servers to defer.
Attached is a log excerpt and backtrace from Debian's exim4-daemon-light 4.96~RC0-1