Postconf segfaults every 5 minutes

[Ari Sovijärvi]

[Impact]

Postfix segfaults when it can't read a map file.

Any users without access to the file will get a core dump when running postfix.

This fix checks that the file was successfully opened before doing any other checks.

[Test Case]

$ lxc launch ubuntu:bionic tester
$ lxc exec tester bash
# apt dist-upgrade
# apt install -y postfix
# touch /etc/postfix/valiases.cf
# chmod 0600 /etc/postfix/valiases.cf
# echo "virtual_alias_maps = pgsql:/etc/postfix/valiases.cf" >> /etc/postfix/main.cf
# su - ubuntu

This command, run as user "ubuntu", will segfault:

$ /usr/sbin/postconf virtual_alias_map

[Regression Potential]

The only change is to short-circuit the error checking code early if the file cannot be opened for any reason. This should not cause regressions, as the alternative is to crash when it attempts to read a bad file descriptor.

[Original Description]

Fresh install of 18.04 server. Every 5 minutes postconf segfaults:

Mar 5 14:30:05 hostname-here kernel: [ 672.082204] postconf[12975]: segfault at 40 ip 0000564d613ff053 sp 00007ffc39e19b90 error 4 in postconf[564d613e7000+25000]
Mar 5 14:30:06 hostname-here kernel: [ 672.303499] postconf[13004]: segfault at 40 ip 000055b29d0f8053 sp 00007fff72f4b740 error 4 in postconf[55b29d0e0000+25000]

According to Apport log, the crash is caused by following command line:

postconf -h queue_directory

Running the command in shell however works as expected and lists the default spool directory (/var/spool/postfix).

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: postfix 3.3.0-1
ProcVersionSignature: Ubuntu 4.15.0-10.11-generic 4.15.3
Uname: Linux 4.15.0-10-generic x86_64
ApportVersion: 2.20.8-0ubuntu10
Architecture: amd64
Date: Mon Mar 5 14:26:27 2018
SourcePackage: postfix
UpgradeStatus: No upgrade log present (probably fresh install)

[Andreas Hasenack (ahasenack)]

Thanks for filing this bug in Ubuntu.

Do you know what is making this call every 5min? I couldn't locate a cronjob about it.

You seem to have other services also failing to start:

-- Logs begin at Tue 2018-02-27 14:59:49 EET, end at Mon 2018-03-05 14:25:05 EET. --
Mar 05 14:19:03 hostname systemd[1]: Failed to start AppArmor initialization.
Mar 05 14:19:04 hostname systemd[1]: Failed to start AppArmor initialization.
Mar 05 14:19:07 hostname iscsid[1778]: iSCSI daemon with pid=1781 started!
Mar 05 14:19:07 hostname systemd[1]: Failed to start LSB: Start or stop stunnel 4.x (TLS tunnel for network daemons).
Mar 05 14:19:12 hostname systemd[1]: Failed to start Service for snap application canonical-livepatch.canonical-livepatchd.

I will need a few more logs to properly diagnose this. Could you please attach the output of:
- dmesg
- cat /var/log/syslog
- free -h

Thanks

[Ari Sovijärvi (apz-fi)]

The AppArmor error message appears to be related to already reported but, and is unrelated. Stunnel was work-in-progress so that's not a problem. Livepatchd doesn't start because 18.04 doesn't currently have Live patch support yet.

[Ari Sovijärvi (apz-fi)]

The problem leaves no traces in any of the logs, but then again I can't find anything that wouldn't work either. Postfix works just fine. There's no crontab jobs or anything that would fire sending a message or anything, besides sending messages works fine as well, with no error messages. It's just these two messages every 5 minutes.

[Andreas Hasenack (ahasenack)]

Do you have mail in the queue? -> sudo mailq

Do you have another 5min cron job that is perhaps failing and therefore sending an email?

Can you see any ownership and permissions problems in the postfix directories? Try "sudo postfix check", and maybe do a quick manual inspection in /var/spool, /var/lib, /var/mail directories

Do you have users and groups coming from a network source, like ldap, nis, ad, and maybe the postfix user or group is mixed with that?

Can you try running that postconf command as the postfix user, or as root if your previous attempt was as postfix or a regular user? Something like "sudo -u postfix -H postconf -h queue_directory"

Thanks

[Ari Sovijärvi (apz-fi)]

After some "yank out lines until the issue stops" detective work, I think I've traced the issue to be this line in Postfix's main.cf:

virtual_alias_maps = pgsql:/etc/postfix/valiases.cf

valiases.cf is very basic example of PostgreSQL based alias maps configuration:

query = SELECT real_address from tempmail where time_expires>now() and temp_address='%s';
user = username-here
password = password-here
dbname = postfix
hosts = ip-here

Postfix-pgsql is naturally installed and Postfix looks up aliases in the database just fine.

[Ari Sovijärvi (apz-fi)]

I think I got it now.

Postconf can't read valiases.cf as user Postfix. I changed the file ownership and permission to allow group postfix to read it, which makes running postconf as user postfix work now.

Curiously the segfault still happens every 5 minutes.

[Ari Sovijärvi (apz-fi)]

I finally found the actual culprit. It's a Munin-plugin that gets Postfix statistics, it calls postconf to figure out Postfix paths.

While the issue is now clear, I think the symptom is still out there, you can crash postconf if you run it with a user that doesn't have permissions to read every included file in main.cf.

[Launchpad Janitor (janitor)]

[Expired for postfix (Ubuntu) because there has been no activity for 60 days.]

[Andreas Hasenack (ahasenack)]

Agreed, and I can still reproduce this in bionic with postfix 3.3.0-1:
ubuntu@bionic-postfix:~$ postconf virtual_alias_map
Segmentation fault (core dumped)
ubuntu@bionic-postfix:~$ dpkg-query -W postfix
postfix 3.3.0-1
ubuntu@bionic-postfix:~$ ll /etc/postfix/valiases.cf
-rw-r----- 1 root root 169 May 7 14:08 /etc/postfix/valiases.cf
ubuntu@bionic-postfix:~$

[Andreas Hasenack (ahasenack)]

The crash happens in vstream_fileno(fp), because fp is a null pointer:

        if ((fp = vstream_fopen(cf_file, O_RDONLY, 0)) == 0
            && errno != EACCES) {
            msg_warn("open \"%s\" configuration \"%s\": %m",
                     dp->db_type, cf_file);
            myfree(dict_spec);
            return;
        }
        if (fstat(vstream_fileno(fp), &st) == 0 && !S_ISREG(st.st_mode)) {
            msg_warn("open \"%s\" configuration \"%s\": not a regular file",
                     dp->db_type, cf_file);
            myfree(dict_spec);
            (void) vstream_fclose(fp);
            return;
        }

vstream_fopen() did return 0 and errno was set to EACCES, I don't know why that check is excluding the permission problem. I emailed upstream (couldn't find a bugtracker for it).

[Ari Sovijärvi (apz-fi)]

I think the only contact to the developers is their postfix-devel
mailing list here:

http://www.postfix.org/lists.html

[Andreas Hasenack (ahasenack)]

And this is new in 3.3.0. I was also looking for some git/svn/bzr/cvs repository to check the history of these changes, but couldn't find it.

[Scott Kitterman (kitterman)]

On Monday, May 07, 2018 03:14:56 PM you wrote:
> vstream_fopen() did return 0 and errno was set to EACCES, I don't know
> why that check is excluding the permission problem. I emailed upstream
> (couldn't find a bugtracker for it).

There is no postfix bugtracker upstream. Did you mail the postfix-users
mailing list? I don't recall seeing this. That's the normal venue for
reporting issues.

Scott K

[Scott Kitterman (kitterman)]

On Monday, May 07, 2018 03:43:26 PM you wrote:
> I think the only contact to the developers is their postfix-devel
> mailing list here:
>
> http://www.postfix.org/lists.html

postfix-devel is generally for developers of postfix. Users should use
postfix-users (even for code issues like this). The postfix developers are
active on postfix-users.

Scott K

[Scott Kitterman (kitterman)]

On Monday, May 07, 2018 04:47:07 PM you wrote:
> And this is new in 3.3.0. I was also looking for some git/svn/bzr/cvs
> repository to check the history of these changes, but couldn't find it.

This is maintained by one of the primary postfix developers:

https://github.com/vdukhovni/postfix

It's the best you'll find publicly.

Scott K

[Andreas Hasenack (ahasenack)]

Thanks Scott, I did email postfix-users with subject "NULL pointer deref in pcf_check_dbms_client() with unreadable map file", but I'm not subscribed. Checking the archive at https://marc.info/?l=postfix-users&r=1&b=201805&w=2 doesn't show my message, so I guess I should subscribe. Or perhaps it's just held up for moderation?

[Scott Kitterman (kitterman)]

On Monday, May 07, 2018 08:56:52 PM you wrote:
> Thanks Scott, I did email postfix-users with subject "NULL pointer deref
> in pcf_check_dbms_client() with unreadable map file", but I'm not
> subscribed. Checking the archive at https://marc.info/?l=postfix-> users&r=1&b=201805&w=2 doesn't show my message, so I guess I should
> subscribe. Or perhaps it's just held up for moderation?

I think you need to be subscribed.

Scott K

[Launchpad Janitor (janitor)]

This bug was fixed in the package postfix - 3.3.0-1ubuntu1

---------------
postfix (3.3.0-1ubuntu1) cosmic; urgency=medium

  * debian/patches/fix-postconf-segfault.diff: Fix a postconf segfault
    when map file cannot be read. Thanks to Viktor Dukhovni <postfix-
    <email address hidden>>. (LP: #1753470)

 -- Andreas Hasenack <email address hidden> Wed, 09 May 2018 09:51:09 -0300

[Matthias Lohr (mlohr)]

Anything new here? Still experiencing this problem.

[Andreas Hasenack (ahasenack)]

It's not forgotten, in fact, I have the ideal candidate to work on this bug :)

[Robie Basak (racb)]

Hello Ari, or anyone else affected,

Accepted postfix into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/postfix/3.3.0-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Simon Déziel (sdeziel)]

SRU verification on Bionic:

root@bp1:~# dpkg -l| grep postfix
ii postfix 3.3.0-1 amd64 High-performance mail transport agent
root@bp1:~# sudo -u ubuntu /usr/sbin/postconf virtual_alias_map
Segmentation fault

root@bp1:~# apt-get -qq update && apt-get install -V postfix
...
The following packages will be upgraded:
   postfix (3.3.0-1 => 3.3.0-1ubuntu0.1)
...
root@bp1:~# sudo -u ubuntu /usr/sbin/postconf virtual_alias_map
/usr/sbin/postconf: warning: virtual_alias_map: unknown parameter

So no more segfault! Thanks

[Launchpad Janitor (janitor)]

This bug was fixed in the package postfix - 3.3.0-1ubuntu0.1

---------------
postfix (3.3.0-1ubuntu0.1) bionic; urgency=medium

  * debian/patches/fix-postconf-segfault.diff: Fix a postconf segfault
    when map file cannot be read. Thanks to Viktor Dukhovni <postfix-
    <email address hidden>>. (LP: #1753470)

 -- Karl Stenerud <email address hidden> Thu, 16 Aug 2018 11:33:00 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for postfix has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Scott Kitterman (kitterman)]

FYI, this is fixed in 3.3.1 upstream, so when that lands, this diff can be dropped.