[SRU] after upgrade to 20.04: dane support is not working

[Jan Büren]

[Impact]

Users cannot send emails using dane-only policy in Focal.

In this SRU we are proposing a microrelease update from version 3.4.10 to 3.4.13 since the changes are self contained. Moreover, there is a Postfix SRU exception which allows microreleases if the bug is fixed in the current development series:

https://wiki.ubuntu.com/StableReleaseUpdates#Postfix

And according to the described process there is no need to define a Test Case and a Regression Potential sections. Upstream has been doing a good work regarding those stable version bug fixes.

Here is the upstream changelog change between 3.4.10 and 3.4.13:

20200416

 Workaround for broken builds after an incompatible change
 in GCC 10. Files: makedefs, Makefile.in.

 Workaround for broken DANE support after an incompatible
 change in GLIBC 2.31. This avoids the need for new options
 in /etc/resolv.conf. Files: dns/dns.h, dns/dns_lookup.c.

20200419

 Bugfix: segfault in the tlsproxy client role when the server
 role was disabled. This typically happens on systems that
 do not receive mail, after configuring connection reuse for
 outbound TLS. Found during program maintenance. File:
 tlsproxy/tlsproxy.c.

20200420

 Noise suppression: shut up a compiler that special-cases
 string literals. Viktor Dukhovni. File milter/milter.c.

20200422

 Security: disable DANE support on Alpine Linux because
 libc-musl provides no indication whether DNS responses are
 authentic. This broke DANE support without a clear explanation.
 File: makedefs.

20200505

 Noise suppression: shut up a compiler that special-cases
 string literals. Viktor Dukhovni. File smtpd/smtpd_check.c.

20200509

 Bugfix (introduced: Postfix 3.5): maillog_file_rotate_suffix
 default value used the minute instead of the month. Reported
 by Larry Stone. Files: conf/postfix-tls-script,
 proto/MAILLOG_README.html, proto/postconf.proto.
 global/mail_params.h, postfix/postfix.c.

20200510

 Bitrot: avoid U_FILE_ACCESS_ERROR after chroot(), by
 initializing the ICU library before making the chroot()
 call. Files: util/midna_domain.[hc], global/mail_params.c.

20200511

 Noise suppression: avoid "SSL_Shutdown:shutdown while in
 init" warnings. File: tls/tls_session.c.

20200515

 Bugfix (introduced: Postfix 2.2): a TLS error for a PostgreSQL
 client caused a false 'lost connection' error for an SMTP
 over TLS session in the same Postfix process. Reported by
 Alexander Vasarab, diagnosed by Viktor Dukhovni. File:
 tls/tls_bio_ops.c.

 Bugfix (introduced: Postfix 2.8): a TLS error for one TLS
 session may cause a false 'lost connection' error for a
 concurrent TLS session in the same tlsproxy process. File:
 tlsproxy/tlsproxy.c.

20200530

 Bugfix (introduced: Postfix 3.1): "postfix tls deploy-server-cert"
 did not handle a missing optional argument. File:
 conf/postfix-tls-script.

20200610

 Bugfix (introduced: Postfix 3.4): in the Postfix SMTP server,
 the SNI callback reported an error when it was called a
 second time. This happened after the server-side TLS engine
 sent a TLSv1.3 HelloRetryRequest (HRR) to a remote SMTP
 client. Reported by Ján Máté, fixed by Viktor Dukhovni.
 File: tls/tls_misc.c.

This new microrelease fixes the dane issue and the build against GCC 10 which makes us drop a patch applied in version 3.4.7-1 (80_glibc2.30-ftbfs.diff).

[Original Description]

My postfix configuration uses dane-only policies for some domains.
After upgrading from LTS 18.04 to the current developing LTS 20.04 this stopped working.

Compare the following commands:

Ubuntu 18.04:

$ posttls-finger -t30 -T180 -c -L verbose,summary bueren.space

posttls-finger: initializing the client-side TLS engine
posttls-finger: using DANE RR: _25._tcp.www.bueren.space IN TLSA 3 0 1 D7:BC:71:07:19:28:E7:97:F9:86:52:02:EB:90:99:4B:B1:DB:EE:8D:FF:B5:D5:6D:15:B2:D8:AC:25:99:AA:5F
posttls-finger: setting up TLS connection to www.bueren.space[31.15.68.4]:25

Ubuntu 20.04:

$ posttls-finger -t30 -T180 -c -L verbose,summary bueren.space

posttls-finger: initializing the client-side TLS engine
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: problem talking to server private/tlsmgr: No such file or directory
posttls-finger: warning: no entropy for TLS key generation: disabling TLS support

Sending email to this domains stopped working with the following (obviously wrong) error message in mail.log:

to=<email address hidden>, relay=none, delay=2126, delays=2126/0.01/0/0, dsn=4.7.5, status=deferred (non DNSSEC destination)

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: postfix 3.4.10-1
ProcVersionSignature: Ubuntu 5.4.0-18.22-generic 5.4.24
Uname: Linux 5.4.0-18-generic x86_64
ApportVersion: 2.20.11-0ubuntu21
Architecture: amd64
Date: Wed Mar 25 11:22:11 2020
EtcMailname: mail.kivitendo.de
Hostname: www.kivitendo.de
InstallationDate: Installed on 2016-12-14 (1196 days ago)
InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.3)
PostconfMydomain: kivitendo-erp.de
PostconfMyhostname: www.kivitendo-erp.de
PostconfMyorigin: /etc/mailname
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ResolvConf:
 # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
 # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
 nameserver 127.0.0.1
 nameserver 127.0.0.1
 search kivitendo-erp.de
SourcePackage: postfix
UpgradeStatus: Upgraded to focal on 2020-03-02 (23 days ago)

[Paride Legovini (paride)]

Thanks Jan for this bug report. I can indeed reproduce the issue; using LXD containers the easiest steps are the following:

1. Launch a clean Eoan container
2. `apt install postfix` with "Local only" config, accept all the defaults
3. Run: `posttls-finger -c gmail.com`. The TLS connection succeeds. The result will be a "certificate verification failed [...] untrusted issuer" but this is because a trust anchor was not setup (I think). The tool works as expected

4. Launch a clean Focal container and install postfix in the same way.
5. Again, run: `posttls-finger -c gmail.com`. The output is:

root@paride-f:~# posttls-finger -c gmail.com
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: problem talking to server private/tlsmgr: No such file or directory
posttls-finger: warning: no entropy for TLS key generation: disabling TLS support

which is clearly wrong.

The postfix package is a sync from Debian, but unfortunately Debian sid is ahead of it, while Debian Buster is behind it, so I can't immediately test how the same version of the package behaves on Debian. However the problem is *not* present in Debian sid or Buster, nor I could find a Debian bug referencing to it.

[Paride Legovini (paride)]

Bare openssl as in:

$ openssl s_client -showcerts -starttls smtp -connect gmail-smtp-in.l.google.com:25

works fine.

[Andreas Hasenack (ahasenack)]

stracing posttls-finger:

4543 connect(3, {sa_family=AF_UNIX, sun_path="private/tlsmgr"}, 110) = -1 ENOENT (No such file or directory)

If you cd into /var/spool/postfix and then try again, it will find the tlsmgr socket. Further I cannot test as it seems my ISP blocks outbound 25/tcp, but in any case I'd suggest to show actual failures from the postfix logs, as this posttls-finger tool is flagged as "an unsupported test program" and might have quirks, like the above (which could still be a bug, i.e, not prefixing the socket path with /var/spool/postfix).

[Scott Kitterman (kitterman)]

I don't recall seeing this issue in Debian, so I don't have any particular
suggestions. Since the postfix is identical, I'd be inclined to look
elsewhere.

Scott K

[Jan Büren (kivitendo)]

Thanks for your replies.

@andreas:
Well, it was a bit hidden in my bug report but the real issue is that postfix doesn't delivers mail to dane-only domains:

to=<email address hidden>, relay=none, delay=2126, delays=2126/0.01/0/0, dsn=4.7.5, status=deferred (non DNSSEC destination)

I created one test account you may use to send some local mail to: <email address hidden>

This is valid DANE domain and to reproduce the issue use the following tls policies:

smtp_tls_policy_maps = hash:/etc/postfix/tls_policy

$ cat /etc/postfix/tls_policy
bueren.space dane-only

The smtp local client tries to verifiy the TLSA entries by using DNSSEC.
I simply use a local unbound DNS server.

This setting stopped working after the upgrade. Maybe the posttls-finger is not so important, but this will trouble all mail admins who have some dane-only entries in their policy (Oops, my DNS Server DNSSEC is bogus -> Nope. Probably the other mail server isn't DANE safe anymore -> Nope.).

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Simon Déziel (sdeziel)]

Comparing strace between Ubuntu and Debian (lxc launch images:debian/10) showed that Debian's version doesn't try to connect to the tlsmgr socket for some reason.

Ubuntu 3.4.10-1:

# grep connect /tmp/strace | grep AF_UNIX
connect(3, {sa_family=AF_UNIX, sun_path="private/tlsmgr"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="private/tlsmgr"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path=@"userdb-523239b8d3d1ae62cb3fa3fd342937ab"}, 42) = -1 ECONNREFUSED (Connection refused)
connect(4, {sa_family=AF_UNIX, sun_path="/run/systemd/userdb/io.systemd.DynamicUser"}, 45) = 0
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)

Debian 3.4.8-0+10debu1:

# grep connect /tmp/strace | grep AF_UNIX
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_UNIX, sun_path="/run/dbus/system_bus_socket"}, 30) = 0

Those with TCP/25 outbound blocked, you can hit my server on port 465 in 'wrapped' mode (-w):

posttls-finger -t30 -T180 -c -L verbose,summary -w smtp.sdeziel.info:465

[Scott Kitterman (kitterman)]

On Thursday, March 26, 2020 12:22:20 PM EDT you wrote:
> Comparing strace between Ubuntu and Debian (lxc launch images:debian/10)
> showed that Debian's version doesn't try to connect to the tlsmgr socket
> for some reason.
>
> Ubuntu 3.4.10-1:
...
> Debian 3.4.8-0+10debu1:
...

If you use buster-proposed-updates you can get 3.4.10 for Debian 10.

Scott K

[Simon Déziel (sdeziel)]

On 2020-03-26 2:40 p.m., Scott Kitterman wrote:
> On Thursday, March 26, 2020 12:22:20 PM EDT you wrote:
>> Comparing strace between Ubuntu and Debian (lxc launch images:debian/10)
>> showed that Debian's version doesn't try to connect to the tlsmgr socket
>> for some reason.
>>
>> Ubuntu 3.4.10-1:
> ...
>> Debian 3.4.8-0+10debu1:
> ...
>
> If you use buster-proposed-updates you can get 3.4.10 for Debian 10.

Thanks, same result though. Debian with 3.4.10-0+deb10u1 works and
doesn't attempt any connection to private/tlsmgr's socket.

[Scott Kitterman (kitterman)]

There are some differences between stable and what was in unstable that Ubuntu
synced.

Does applying this change help:

https://salsa.debian.org/postfix-team/postfix-dev/-/commit/
b8e0b846e34eeaaa2315ead2304824b21b01fe7a

Scott K

[Simon Déziel (sdeziel)]

On 2020-03-26 3:54 p.m., Scott Kitterman wrote:
> Does applying this change help:
>
> https://salsa.debian.org/postfix-team/postfix-dev/-/commit/
> b8e0b846e34eeaaa2315ead2304824b21b01fe7a

Does not help.

Sion

[Scott Kitterman (kitterman)]

On Thursday, March 26, 2020 5:31:05 PM EDT you wrote:
> On 2020-03-26 3:54 p.m., Scott Kitterman wrote:
> > Does applying this change help:
> >
> > https://salsa.debian.org/postfix-team/postfix-dev/-/commit/
> > b8e0b846e34eeaaa2315ead2304824b21b01fe7a
>
> Does not help.
>
> Sion

OK. That's the only even tangentially relevant difference between Debian 10
where it works and Ubuntu F where it doesn't.

I've got no further ideas.

Scott K

[Nick Tait (nick.t)]

I have this issue also, but my set-up involves having my submission server send all emails to an internal mail relay, which then sends to the destination SMTP server. I'm using "dane-only" for the connection from the submission server to the mail relay, meaning that it is used for all external outgoing mail. So I'm seeing the same same behaviour as described above, but the deferred message is slightly different:

to=<email address hidden>, relay=none, delay=0.13, delays=0.1/0.02/0.01/0, dsn=4.7.5, status=deferred (no TLSA records found)

I can only assume that the different message is due to a difference such as not using MX look-up to send to the mail relay? But I thought I'd mention it in case it is relevant?

And just to confirm, yes this problem started after I performed the dist-upgrade to Focal Fossa. FYI As a workaround I've changed to using "encrypt" instead of "dane-only".

I'm happy to provide more detailed information (e.g. configuration files) if it would help?

Thanks,
Nick.

[Jan Büren (kivijan)]

Hi Nick,
yep. My workaround was to change this to dane instead of dane-only.

It is a tiny bit disappointing that it is still broken because I prefer to use ubuntu as a server system instead of debian.
Reading this discussion I guess the feature is working in debian but not in ubuntu.

Additionally I resisted building Postfix from source to detect the deeper bug reason but if this bug doesn't process maybe this would be an option.

I enjoyed the idea from this blogpost: https://briefings.threeletter.agency/post/120317637998/fixing-postfix-a-trip-down-a-smtp-tls-rabbit

[Nick Tait (nick.t)]

Hi Jan.

Thanks for sending me that blogpost. I found it a really good read! :-)

I tried downloading some open-source code one other time (can't actually
remember what it was), and got horribly confused, but maybe (like you) I
might feel inspired to have another crack at some stage...

Nick.

On 19/05/20 10:25 pm, Jan Büren wrote:
> Hi Nick,
> yep. My workaround was to change this to dane instead of dane-only.
>
> It is a tiny bit disappointing that it is still broken because I prefer to use ubuntu as a server system instead of debian.
> Reading this discussion I guess the feature is working in debian but not in ubuntu.
>
> Additionally I resisted building Postfix from source to detect the
> deeper bug reason but if this bug doesn't process maybe this would be an
> option.
>
> I enjoyed the idea from this blogpost:
> https://briefings.threeletter.agency/post/120317637998/fixing-postfix-a
> -trip-down-a-smtp-tls-rabbit
>

[Nick Tait (nick.t)]

Hi guys.

I've made some progress with narrowing down the cause of this problem. It looks like the whole "private/tlsmgr" thing is a bit of a red herring, and the actual problem seems to be that the DNS query that is sent lacks the AD flag.

I've confirmed that RES_USE_DNSSEC is included in the flags when the DNS functions are called within the code, so I'm suspecting that systemd.resolvd has something to do with it. E.g. I've just noticed that when I run "systemd-resolve --status", the output includes:

    DNSSEC supported: no

Which looks like a pretty good reason for this behaviour.

Tomorrow I'll try changing the systemd.resolvd settings to enable DNSSEC and see if that makes a difference.

Nick.

[Jan Büren (kivijan)]

HI Nick,
nice idea, sounds reasonable, but ...

I am not using systemd.resolv and I can see the ad flag if I query unbound locally.
I am simply guessing: Maybe tlsmgr really needs systemd?

Anyway here are my settings and snippets:

 sudo systemctl disable systemd-resolved
 sudo systemctl enable unbound-resolvconf

 # dig +dnssec A www.dnssec.cz | grep ad
 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

Taking a short look into the package-helper for unbound-resolvconf I don't see anything obvious.

I changed the settings after or because of reading this blog:

https://blobfolio.com/2017/05/fix-linux-dns-issues-caused-by-systemd-resolved/

[Paride Legovini (paride)]

I still think Nick's clue is promising, let's see what the results of his tests will be.

There could be more than one thing causing DANE not to work. If enabling DNSSEC in systemd-resolved makes it work we'll try to figure out why it doesn't work with unbound-resolvconf.

[Nick Tait (nick.t)]

Sorry for the delay in providing an update...

My theory about the systemd-resolved DNSSEC option proved incorrect.

So taking a step back, here is what I know about the problem that I am observing (i.e. specific to my test set-up):

* posttls-finger makes three DNS queries: The first query is for an MX record (which returns NODATA), then an A record (which returns a valid IP address), and finally a AAAA record (which returns NODATA).
* None of these include the AD flag in the request (question) - which is different from the behaviour seen when using 'dig' - but as it turns out, you don't need to include AD in the request to get AD in the reply. So this wasn't the problem.
* All three queries include the DO flag (to request RRSIG records).
* Using Wireshark I can see that the DNS response coming back to the 3 queries includes the AD flag (and the DO flag, and also RRSIG/NSEC3 records as appropriate).
* However in spite of this, the DNS results returned from the C functions indicate AD=0 - i.e. they are saying that AD flag isn't present in the response, even though (from the Wireshark trace) it clearly is.
* Based on this, my hypothesis is that the issue is with the C functions (libresolv).

So my plan now is to write a C program to test this theory, which will make a DNS query and then check the AD flag. If my theory is correct then running the same program on Ubuntu 20.04 and an older distribution should give different results.

Nick.

[Nick Tait (nick.t)]

I've created a C program (attached) which demonstrates the issue. It can be compiled using the following command:

gcc dnsadtest.c -lresolv

This command generates an executable called "a.out".

If I run a.out on Ubuntu 18.04 (bionic) I get the following output:

length of answer = 251
id = 10740
opcode = 0
rcode = 0
rd flag = 1
tc flag = 0
aa flag = 0
qr flag = 1
cd flag = 0
ad flag = 1
ra flag = 1
qdcount = 256
ancount = 512
nscount = 0
arcount = 256

You can see that the output includes "ad flag = 1".

But if I run a.out on Ubuntu 20.04 (focal) I get the following output:

length of answer = 251
id = 48769
opcode = 0
rcode = 0
rd flag = 1
tc flag = 0
aa flag = 0
qr flag = 1
cd flag = 0
ad flag = 0
ra flag = 1
qdcount = 256
ancount = 512
nscount = 0
arcount = 256

And you can see that the output includes "ad flag = 0".

Based on this I believe there is a bug in the DNS resolver in Ubuntu 20.04.
Unfortunately my 18.04 server is 32-bit and by 20.04 server is 64-bit, but my testing gives the following results:

18.04 32-bit = libc6:i386 2.27-3ubuntu1 = works (ad flag = 1)
20.04 64-bit = libc6:amd64 2.31-0ubuntu9 = exhibits the issue (ad flag = 0)
20.04 32-bit* = libc6-i386 2.31-0ubuntu9 = exhibits the issue (ad flag = 0)
* = running a.out compiled on 32-bit machine

So it looks to me like the problem was introduced in the libc6 package somewhere between version 2.28 and 2.31 (inclusive).

Can anyone corroborate my findings?

Thanks,
Nick.

[Scott Kitterman (kitterman)]

Re-assigning to glibc then. I think that's a good demonstration this isn't a postfix issue.

[Nick Tait (nick.t)]

I (by accident) discovered that glibc has introduced a new resolver option in resolv.h:

#define RES_TRUSTAD 0x04000000 /* Request AD bit, keep it in responses. */

I've done some testing with this, and it resolves the issue with the AD flag not being returned.

So based on this I think this bug needs to be changed back to postfix, and postfix needs to be updated to include this flag? Ideally the behaviour require should be:

* If RES_TRUSTAD is defined, then postfix should use that instead of RES_USE_DNSSEC and RES_USE_EDNS0.
* If RES_TRUSTAD is not defined, then postfix should maintain current behaviour of using RES_USE_DNSSEC and RES_USE_EDNS0.

If the above is implemented it would reduce the size of the DNS queries, because they won't include the RRSIG records that "come for free" when the DO bit is set (based on RES_USE_DNSSEC).

Thanks,
Nick.

[Paride Legovini (paride)]

Thanks Nick for your excellent debugging work. I found two long threads related to this issue in the postfix-users mailing list:

http://postfix.1071664.n5.nabble.com/Outgoing-DANE-not-working-tt105397.html#none

http://postfix.1071664.n5.nabble.com/PATCH-Glibc-2-31-DNSSEC-and-GCC-10-tt105461.html#none

What happens if you add:

 options trust-ad

to your resolv.conf?

[Scott Kitterman (kitterman)]

That change is included in postfix 3.4.11 which is, of course, just after the one that's in 20.04. "Someone" (i.e. not me) should look into an SRU to update postfix. It has an existing microversion release authorization from the Tech Board, so it shouldn't be too hard, it just takes someone involved in Ubuntu development to turn the crank on the process.

[Lucas Kanashiro (lucaskanashiro)]

After Scott's comment about updating postfix to 3.4.11 I checked the changelog of this version and I noticed the only change from 3.4.10 is:

20200416

 Workaround for broken builds after an incompatible change
 in GCC 10. Files: makedefs, Makefile.in.

 Workaround for broken DANE support after an incompatible
 change in GLIBC 2.31. This avoids the need for new options
 in /etc/resolv.conf. Files: dns/dns.h, dns/dns_lookup.c.

I marked Groovy as Fix Released because the version 3.5.2 also contains this commit. However, when I tried the mentioned commands in a Groovy container I faced the same issue:

$ lxc launch ubuntu-daily:groovy postfix-dane-issue
$ lxc shell postfix-dane-issue
# apt install postfix
# dpkg -l | grep postfix
ii postfix 3.5.2-1 amd64 High-performance mail transport agent
# posttls-finger -c gmail.com
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: problem talking to server private/tlsmgr: No such file or directory
posttls-finger: warning: no entropy for TLS key generation: disabling TLS support
^C
# posttls-finger -t30 -T180 -c -L verbose,summary -w smtp.sdeziel.info:465
posttls-finger: initializing the client-side TLS engine
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: problem talking to server private/tlsmgr: No such file or directory
posttls-finger: warning: no entropy for TLS key generation: disabling TLS support
posttls-finger: warning: lost connection while sending QUIT command

As Paride mentioned those commands work without any problem in Eoan and also in Debian unstable which has the same postfix version:

$ lxc launch images:debian/sid postfix-dane-issue-debian
$ lxc shell postfix-dane-issue-debian
# apt install postfix
# dpkg -l | grep postfix
ii postfix 3.5.2-1+b1 amd64 High-performance mail transport agent
# posttls-finger -c gmail.com
posttls-finger: Failed to establish session to gmail.com via gmail-smtp-in.l.google.com: connect to gmail-smtp-in.l.google.com[2800:3f0:4003:c00::1a]:25: Connection timed out
^C
# posttls-finger -t30 -T180 -c -L verbose,summary -w smtp.sdeziel.info:465
posttls-finger: initializing the client-side TLS engine
posttls-finger: setting up TLS connection to smtp.sdeziel.info[2001:470:b1c3:7942::25]:465
posttls-finger: smtp.sdeziel.info[2001:470:b1c3:7942::25]:465: TLS cipher list "aNULL:-aNULL:HIGH:MEDIUM:+RC4:@STRENGTH:!aNULL"
posttls-finger: smtp.sdeziel.info[2001:470:b1c3:7942::25]:465: depth=1 verify=0 subject=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
posttls-finger: smtp.sdeziel.info[2001:470:b1c3:7942::25]:465: depth=0 verify=1 subject=/CN=smtp.sdeziel.info
posttls-finger: certificate verification failed for smtp.sdeziel.info[2001:470:b1c3:7942::25]:465: untrusted issuer /O=Digital Signature Trust Co./CN=DST Root CA X3
posttls-finger: smtp.sdeziel.info[2001:470:b1c3:7942::25]:465: subject_CN=smtp.sdeziel.i...

[Scott Kitterman (kitterman)]

On Monday, June 8, 2020 4:59:30 PM EDT you wrote:
> postfix (3.5.2-1+b1) sid; urgency=low, binary-only=yes
> .
> * Binary-only non-maintainer upload for amd64; no source changes.
> * Rebuild against libicu67
> .
> -- all / amd64 / i386 Build Daemon (x86-conova-01)
> <email address hidden> Wed, 03 Jun 2020 20:54:57
> +0000
>
> Do we need this in Groovy?

No. It's unrelated. Ubuntu does source uploads when rebuilds are needed in
any case.

[Lucas Kanashiro (lucaskanashiro)]

That was my initial thought. So I am not sure if just a postfix update to version 3.4.11 (or 3.5.2 in Groovy) fixes the mentioned issue.

FWIW, I have a PPA here with the version 3.4.11 built in Focal and the behavior is the same as version 3.5.2 in Groovy:

https://launchpad.net/~lucaskanashiro/+archive/ubuntu/focal-postifx-lp1868955/

[Scott Kitterman (kitterman)]

In that case, someone with the new libc and postfix 3.4.11/3.5.2 or later
should write to the postfix-users list and ask for help. My recollection is
that the upstream developers did not have access to a system with the new libc
and were dependent on third party testing, so something may have been missed.

[Lucas Kanashiro (lucaskanashiro)]

@Nick and @Jan: could you please try postfix 3.4.11 available in the PPA I mentioned above and give us some feedback? I'd like to confirm that before asking for help on postfix-users mailing list.

[Jan Büren (kivitendo)]

Hi Lucas,
yes your ppa version seems to work.
I can also send emails again with the dane-only policy.

Details:

The warning still exists, but posttls-finger gets a valid RR record:

root@www:~# posttls-finger -t30 -T180 -c -L verbose,summary bueren.space
posttls-finger: initializing the client-side TLS engine
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: problem talking to server private/tlsmgr: No such file or directory
posttls-finger: warning: no entropy for TLS key generation: disabling TLS support
posttls-finger: using DANE RR: _25._tcp.www.bueren.space IN TLSA 3 0 1 D7:BC:71:07:19:28:E7:97:F9:86:52:02:EB:90:99:4B:B1:DB:EE:8D:FF:B5:D5:6D:15:B2:D8:AC:25:99:AA:5F

[Jan Büren (kivitendo)]

You can foolproof this with your settings by writing to my test mail account and setting this domain to dane-only:

"I created one test account you may use to send some local mail to: <email address hidden>"

[Lucas Kanashiro (lucaskanashiro)]

Good to hear that Jan. I might have messed up my local setup. I'll proceed with the SRU process to make this fix land in Focal.

[Nick Tait (nick.t)]

Hi guys.

Sorry for my slow response.

Jan what you are seeing is consistent with what I expected. i.e. The addition of the RES_TRUSTAD flag to the postfix code allows the DANE mechanism to work.

During my debugging of the postfix code, I was using posttls-finger, but I never encountered the issue with the "posttls-finger: warning: connect to private/tlsmgr: No such file or directory" message at all. But of course when I'm running posttls-finger as part of my postfix installation I do see that message (BTW I'm still on the unpatched version at this moment).

I'm not sure why the behaviour is different, but it does seem that these are two different problems.

Nick.

[Stefan Voelkel (stefan-voelkel)]

FWIW, I found this issue when debugging SSHFP DNS Records not working on Linux Mint 20.

In my case, setting 'options trust-ad' in /etc/resolv.conf helped, see 446997ff1433d33452b81dfa9e626b8dccf101a4 in git://sourceware.org/git/glibc.git

[Lucas Kanashiro (lucaskanashiro)]

Please do not consider version 3.4.11-0ubuntu1. I am preparing a SRU to include version 3.4.13 which also fixes LP #1881196.

[Robie Basak (racb)]

Thanks, Scott, for all your help and advice on this - both here and in the other bug.

[Robie Basak (racb)]

Hello Jan, or anyone else affected,

Accepted postfix into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/postfix/3.4.13-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Lucas Kanashiro (lucaskanashiro)]

Hi Nick and Jan (and anyone affected by this bug),

Could you please try out version 3.4.13-0ubuntu1 in focal-proposed? And tell us if it's properly fixed for the sake of the SRU process? If it is the case please remove verification-needed and verification-needed-focal tags from this bug and add verification-done and verification-done-focal.

Thank you!

[Nick Tait (nick.t)]

Hi Lucas.I'll give it a crack tomorrow and update the case with the results.Thanks, Nick.
null

[Jan Büren (kivitendo)]

Hi,
I added the proposed repo and it works as expected.

Jun 27 10:49:26 www postfix/master[1920]: daemon started -- version 3.4.13, configuration /etc/postfix

root@www:~# posttls-finger -t30 -T180 -c -L verbose,summary bueren.space
posttls-finger: initializing the client-side TLS engine
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: connect to private/tlsmgr: No such file or directory
posttls-finger: warning: problem talking to server private/tlsmgr: No such file or directory
posttls-finger: warning: no entropy for TLS key generation: disabling TLS support
posttls-finger: using DANE RR: _25._tcp.www.bueren.space IN TLSA 3 0 1 D7:BC:71:07:19:28:E7:97:F9:86:52:02:EB:90:99:4B:B1:DB:EE:8D:FF:B5:D5:6D:15:B2:D8:AC:25:99:AA:5F

[Jan Büren (kivitendo)]

I am not sure how to tag this bug (fix-verified).

In the above dialog I can only set the state to fix-released ...

[Nick Tait (nick.t)]

Hi everyone.

I too have tested the proposed postfix release, and it fixes the DANE issue and also passed all my regression tests. :-)

There is still an issue with posttls-finger attempting to connect to private/tlsmgr, which fails if it isn't run from /var/spool/postfix as root. (This was actually the symptom originally described by this bug when it was first logged by Jan. I'll raise a separate case for that, and update this case with the bug ID...)

Thanks everyone for your efforts in resolving the DANE issue.

Nick.

[Nick Tait (nick.t)]

FYI I've created the following bug report for the posttls-finger issue:

https://bugs.launchpad.net/ubuntu/+source/postfix/+bug/1885403

Thanks,
Nick.

[Launchpad Janitor (janitor)]

This bug was fixed in the package postfix - 3.4.13-0ubuntu1

---------------
postfix (3.4.13-0ubuntu1) focal; urgency=medium

  * New upstream release: 3.4.13
    - Workaround for broken DANE support after an incompatible change in
      GLIBC 2.31 (LP: #1868955)
    - Fix "postfix tls deploy-server-cert", now it handles a missing optional
      argument (LP: #1881196)
  * Drop patch 80_glibc2.30-ftbfs.diff. This patch is not needed anymore and
    it does not cleanly apply to this new upstream release.

 -- Lucas Kanashiro <email address hidden> Fri, 19 Jun 2020 14:11:03 -0300

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for postfix has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.