Installation fails with fully qualified domains (w/ trailing dot)

[Heinrich Schuchardt]

I have been looking at
https://autopkgtest.ubuntu.com/results/autopkgtest-mantic/mantic/amd64/f/fwlogwatch/20230502_145023_da50b@/log.gz
where the installation of postfix fails in the autopkgtest environment.

The hostname passed to the test is "adt-mantic-amd64-fwlogwatch-20230502-143927-lrg-root5".

After concatenating with the domain in postfix function check_myhostname() the hostname becomes "adt-mantic-amd64-fwlogwatch-20230502-143927-lrg-root5.openstack.prodstack5.lan." which terminates with a dot. Later function valid_hostname() detects that this is invalid.

As the problem does not occur in a local autopkgtest I assume this is due to some issue with the openstack configuration.

Best regards

Heinrich

[Colin Watson (cjwatson)]

Launchpad isn't responsible for running autopkgtests. Reassigning.

[Iain Lane (laney)]

Fully qualified domain names do have a trailing dot, so this is a legal domain name. Question is whether postfix likes it like that, and it seems like not.

I think this setting comes from postfix.postinst. There's some code in there (myfqdn()) which finds out the fqdn of the host. There's a bit of code to look up the domain name from search paths in /etc/resolv.conf. That will strip off a trailing dot if it is there.

Since we still see a trailing dot I think that means that the hostname in /etc/hostname must itself have the trailing dot in it. As far as I can tell that is perfectly fine, if uncommon.

If that's all right (a big assumption) then I would suggest the fix is in postfix.postinst to always strip a trailing dot if it finds one, thereby making myhostname into something which postfix likes the look of.

Do let me know if I misunderstood any of this :-)

[Paride Legovini (paride)]

Hi Iain, you are fully right. There is only one piece of the puzzle that does not fit: when running postfix autopkgtests triggered by postfix itself, myhostname is set to 'localhost', which makes postfix happy. See for example

https://autopkgtest.ubuntu.com/packages/postfix/mantic/amd64

The trailing dot is only present when postfix is installed as a dependency. I still can't figure out why this difference (maybe there's an extra reboot in some cases?).

[Heinrich Schuchardt (xypron)]

This is the reference document for domain names https://www.ietf.org/rfc/rfc1034.txt (DOMAIN NAMES - CONCEPTS AND FACILITIES).

[Paride Legovini (paride)]

hostname(7) also mentions the trailing dot, we agree that's formally valid.

[Paride Legovini (paride)]

This is a valid postfix bug, uncovered by a change in the autopkgtest environment. We'll try to undo the change on the autopkgtest side, but the right way to fix the issue would be fixing postfix.

*Ideally* this should be fixed upstream, making postfix handle FQDNs with a trailing dot in myhostname. However fixing it in packaging is also good (no need for quilt patches). This could be done by making postinst script trim the trailing dot from the domain name before configuring myhostname. The relevant bit is the myfqdn() function in postfix.postinst:

myfqdn() {
    myhostname=$(hostname --fqdn 2>/dev/null || echo "")
    [...]
}

Reproducer:

---------------------------------------------------------
$ lxc launch ubuntu-daily:mantic paride-m
$ lxc exec paride-m bash

# hostname --fqdn
paride-m.lxd
# echo "127.0.1.1 foo.bar. foo" >> /etc/hosts
# hostnamectl hostname foo
# hostname --fqdn
foo.bar.

# apt install postfix
[...]
Running newaliases
newaliases: warning: valid_hostname: misplaced delimiter: foo.bar.
[...]
E: Sub-process /usr/bin/dpkg returned an error code (1)
---------------------------------------------------------

Given that postfix is currently a sync it would be nice to fix this in Debian:

https://salsa.debian.org/postfix-team/postfix-dev/-/blob/d3b9ba0c893fa45d8859e627d43dd3746ba915c7/debian/postfix.postinst#L37

[Paride Legovini (paride)]

Note that we see the autopkgtest failure only when postfix is installed as a (test-)dependency of some other package. Interestingly postfix own autopkgtest does not fail. This is because postfix is not installed as a test dependency, but "manually" by the d/t/postfix script:

# reconfigure postfix
debconf-set-selections <<< "postfix postfix/mailname string localhost" 2>&1
debconf-set-selections <<< "postfix postfix/main_mailer_type string 'Internet Site'" 2>&1

# install and modify
hostname localhost
apt-get install -y postfix 2>&1

[Paride Legovini (paride)]

Autopkgtest bug: LP: #2019472.

[Scott Kitterman (kitterman)]

On Saturday, May 13, 2023 5:23:46 AM EDT Paride Legovini wrote:
> This is a valid postfix bug, uncovered by a change in the autopkgtest
> environment. We'll try to undo the change on the autopkgtest side, but
> the right way to fix the issue would be fixing postfix.

I'm not fully convinced here.

The postfix documentation says myhostname should be the output of gethostbyname
(which is obsolete in POSIX, but that's another question). The DNS RFCs are
about the on the wire format, not local configuration files.

https://www.postfix.org/postconf.5.html#myhostname

man 3 gethostbyname on my Debian Stable system says it's conformant to POSIX.
1-2001 (anyone have a copy)? As far as I can tell based on the man page and
my local experimentation, it does not return a dot on the end.

I think you need to fix the test environment.

Scott K

[Paride Legovini (paride)]

Hi Scott, thanks for chiming it!

We may digress on whether not handling a trailing dot in myhostname is a postfix bug or not, but I think it's going to be a mostly academic discussion. In practice nobody is going to set myhostname to a host with a trailing dot.

What is setting myhostname to a value with a trailing dot is d/postfix.postinst. I think that should be fixed, stripping the trailing dot to myhostname after doing:

  myhostname=$(hostname --fqdn 2>/dev/null || echo "")

as `hostname --fqdn` is meant to return a FQDN which definitely can have a trailing dot. This could be done by something like:

  myhostname=${myhostname%.}

(By grepping debian/* I found 5 occurrences of `hostname --fqdn`, I suspect all would need a similar treatment.)

In other words I think the right thing to do here it to consider this a packaging bug.

[Scott Kitterman (kitterman)]

If someone wants to prepare a merge request, I think it's not an unreasonable change to improve input validation, so I would accept it:

https://salsa.debian.org/postfix-team/postfix-dev

I don't plan to take the time to do it myself. Any upload would happen after the bookworm release.

Note: For the postfix package, I generate d/changelog from the Git commit messages, so please don't include a d/changelog entry in the MR.

[Scott Kitterman (kitterman)]

It's actually more complicated than that (now that I've thought it over a bit more), since non-FQDN myhostname values are supported.

If I have:

mydomain = example.com

and either:

myhostname = mail.example.com
myhostname = mail.

Then the expanded value of myhostname is mail.example.com either way. Per the Postfix documentation, a non-FQDN my hostname gets mydomain appended to it.

Just stripping trailing dots breaks that.

[Paride Legovini (paride)]

I see, thanks. I still stink it's doable in the right way, but as you say with extra care in checking how myhostname and mydomain are constructed.

[Paride Legovini (paride)]

My proposal is:

https://code.launchpad.net/~paride/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/442995

[Iain Lane (laney)]

> myhostname = mail.

That value would not be allowed by postfix

root@lucky-fly:~# postconf -e myhostname=mail. mydomain=example.com
root@lucky-fly:~# newaliases
newaliases: warning: valid_hostname: misplaced delimiter: mail.
newaliases: fatal: file /etc/postfix/main.cf: parameter myhostname: bad parameter value: mail.

This is basically the issue we're dealing with here: the postinst allows the trailing dot through if it gets it from /etc/resolv.conf or `hostname --fqdn`. In the case that it finds "mail.example.com." in there, it would be correct per the documentation to write "mail.example.com" to myhostname, then mydomain would effectively not be used.*

https://sources.debian.org/src/postfix/3.7.5-2/src/postconf/postconf_builtin.c/#L191-L214

* more complicated than this but that's the jist; see append_dot_mydomain and other options, the variable is used if append_dot_mydomain is true and myhostname does not look like a fqdn.

[Mitchell Dzurick (mitchdz)]

Apologies on the delay in this bug, I'm just picking this up now.

It sounds like the consensus is that it's alright for myhostname to have a trailing dot, ONLY if mydomain is properly set? I'd like to make sure I'm on the same page with everyone.

[Paride Legovini (paride)]

Hi Mitchell, I think that's technically correct, but I would be more in favor of always stripping the trailing dot from myhostname. I don't see the downside of doing this: if mydomain is going to be appended to myhostname, a dot will also be added. At least that's how I read the append_dot_mydomain documentation.

[Mitchell Dzurick (mitchdz)]

Hmmm I don't want to rely on append_dot_mydomain as that is a user configurable parameter, and seems to default to no.

This could be fixed for most us by changing debian/postfix.postinst to strip the trailing dot if it exists, but the issue will still be exposed to users who manually set the hostname just like how lain mentioned in #15, so it will be more of a bandaid for our use-case.

Why does postfix not like the trailing dot anyways? It seems legal (if not common) to have (I understand this might not be the right area to have this discussion).

[Mitchell Dzurick (mitchdz)]

I do see https://code.launchpad.net/~paride/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/442995 is merged, so we shouldn't be seeing the error in our autopkgtest environment anymore. The aim of this bug should now be to fix it upstream if that seems reasonable.

[Scott Kitterman (kitterman)]

On Thursday, August 10, 2023 5:44:46 PM EDT you wrote:
> I do see https://code.launchpad.net/~paride/autopkgtest-> cloud/+git/autopkgtest-cloud/+merge/442995 is merged, so we shouldn't be
> seeing the error in our autopkgtest environment anymore. The aim of this
> bug should now be to fix it upstream if that seems reasonable.

If you look earlier in the bug, the current behavior is consistent with the
postfix documentation, so there's nothing to fix.

Scott K

[Mitchell Dzurick (mitchdz)]

> If you look earlier in the bug, the current behavior is consistent with the
> postfix documentation, so there's nothing to fix.

Scott, are you referring to comment #9?

[Scott Kitterman (kitterman)]

On Friday, August 11, 2023 9:36:55 AM EDT you wrote:
> > If you look earlier in the bug, the current behavior is consistent with
> > the
> > postfix documentation, so there's nothing to fix.
>
> Scott, are you referring to comment #9?

Yes,

Scott K

[Mitchell Dzurick (mitchdz)]

Shouldn't postfix improve handling a user setting myhostname with a trailing dot? Or is it just expected to allow the user to set the value and then encounter the error later?

e.g. From Lain's #15 comment

# postconf -e myhostname=mail. mydomain=example.com
# newaliases
newaliases: warning: valid_hostname: misplaced delimiter: mail.
newaliases: fatal: file /etc/postfix/main.cf: parameter myhostname: bad parameter value: mail.

It sounds like in this scenario myhostname should expand to mail.example.com without an error.

However I do agree this is a weird scenario and I doubt any user will set their hostname with a trailing dot intentionally

[Mitchell Dzurick (mitchdz)]

After a short discussion, this bug will be set to wont-fix because it is unlikely users will intentionally set a trailing dot to the hostname, and if they do, it is a relatively simple fix for the problem.

If this bug is impacting you, feel free to comment and this can be re-addressed.