Assertion failure when PID 1 receives a zero-length message over notify socket

[Jorge Niedbalski]

Environment:

Xenial 16.04.1
Amd64

Description.

Systemd fails an assertion in manager_invoke_notify_message when a zero-length message is received over /run/systemd/notify. This allows a local user to perform a denial-of-service attack against PID 1.

How to trigger the bug:

$ while true; do NOTIFY_SOCKET=/run/systemd/notify systemd-notify ""; done

The following entries are written into /var/log/syslog, at this point systemd is crashed.

Sep 28 20:57:20 ubuntu systemd[1]: Started User Manager for UID 1000.
Sep 28 20:57:28 ubuntu systemd[1]: Assertion 'n > 0' failed at ../src/core/manager.c:1501, function manager_invoke_notify_message(). Aborting.
Sep 28 20:57:29 ubuntu systemd[1]: Caught <ABRT>, dumped core as pid 1307.
Sep 28 20:57:29 ubuntu systemd[1]: Freezing execution.

Public bug: https://github.com/systemd/systemd/issues/4234

The original USN/security fix in https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu10 introduced another local DoS due to fd exhaustion:

  NOTIFY_SOCKET=/run/systemd/notify python3 -c 'from systemd import daemon; daemon.notify("", fds=[0]*100)'

Run this a few times and watch "sudo ls -l /proc/1/fd" grow.

[Steve Beattie (sbeattie)]

CVE request: http://www.openwall.com/lists/oss-security/2016/09/28/9

[Steve Beattie (sbeattie)]

FYI, I've pushed xenial and yakkety systemd packages with Jorge's proposed fix from https://github.com/systemd/systemd/pull/4237 in the ubuntu-security-proposed ppa at https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/ for people to test.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 229-4ubuntu10

---------------
systemd (229-4ubuntu10) xenial-security; urgency=medium

  * SECURITY UPDATE: zero-length notify message triggers abort/denial of
    service
    - systemd-dont_assert_on_zero_length_message-lp1628687.patch: change
      assert to simple return + log (LP: #1628687)
    - Thanks to Jorge Niedbalski <email address hidden> for
      the patch.

 -- Steve Beattie <email address hidden> Wed, 28 Sep 2016 14:21:42 -0700

[Martin Pitt (pitti)]

That initial fix just changed a DoS through assert() into a DoS through fd exhaustion. This is being handled in https://github.com/systemd/systemd/pull/4242 .

Please let's handle this upstream first and not put out another USN in haste -- after all, this is just a local DoS, so far from being a catastrophe (you can DoS the machine as user in lots of other ways).

[Martin Pitt (pitti)]

For a follow-up xenial -security update we need:
  https://github.com/systemd/systemd/commit/8523bf7dd5
  https://github.com/systemd/systemd/commit/9987750e7

which I just cherry-picked into the Debian packaging tree (for yakkety too):

  https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?id=64196d509b
  https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?id=c172afdd2

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 231-9

---------------
systemd (231-9) unstable; urgency=medium

  * pid1: process zero-length notification messages again.
    Just remove the assertion, the "n" value was not used anyway. This fixes
    a local DoS due to unprocessed/unclosed fds which got introduced by the
    previous fix. (Closes: #839171) (LP: #1628687)
  * pid1: Robustify manager_dispatch_notify_fd()
  * test/networkd-test.py: Add missing writeConfig() helper function.

 -- Martin Pitt <email address hidden> Thu, 29 Sep 2016 23:39:24 +0200

[Martin Pitt (pitti)]

I added the two patches to the xenial branch, and will test/upload the SRU tomorrow.

[Steve Beattie (sbeattie)]

Martin, if you can point me at the xenial branch, we can push this through the security pocket. I wanted to wait and see if there were any further issues addressed (and to not release an update on a Friday). Thanks!

[Martin Pitt (pitti)]

> Martin, if you can point me at the xenial branch, we can push this through the security pocket.

https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=ubuntu-xenial&id=1aa1f84c
(this just cleans up the original -security update to still work with gbp pq)

https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=ubuntu-xenial&id=a80398c79
https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=ubuntu-xenial&id=0f6614488

I'm fine with landing this as part of the next SRU (I'll get it into -proposed today). A local DoS (the original and this new one) isn't particularly exciting after all. If you rather want to to handle this as a security update, then you can grab these three patches, and I'll rebase the branch/re-do the SRU again.

[Chris Halse Rogers (raof)]

Hello Jorge, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Martin Pitt (pitti)]

I verified that with the -proposed version you cannot create orphaned notify FDs any more in pid 1 using the test case I just added.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 229-4ubuntu11

---------------
systemd (229-4ubuntu11) xenial; urgency=medium

  * 73-usb-net-by-mac.rules: Split kernel command line import line.
    Reportedly this makes the rule actually work on some platforms. Thanks
    Alp Toker! (LP: #1593379)
  * fsckd: Do not exit on idle timeout if there are still clients connected
    (Closes: #788050, LP: #1547844)
  * libnss-*.prerm: Remove possible [key=value] options from NSS modules as
    well. (LP: #1625584)
  * Backport networkd 231. Compared to 229 this has a lot of fixes, some of
    which we need for good netplan support. Backporting them individually
    would be a lot more work and a lot less robust, and we did not use/support
    networkd in 16.04 so far. Drop the other network related patches as they
    are included in this backport now. (LP: #1627641)
  * debian/tests/networkd: Re-enable the the DHCPv6 tests. The DHCPv6
    behaviour is fixed with the above backport now.
  * pid1: process zero-length notification messages again. Just remove the
    assertion, the "n" value was not used anyway. This fixes a local DoS due
    to unprocessed/unclosed fds which got introduced by the previous fix.
    (LP: #1628687)
  * pid1: Robustify manager_dispatch_notify_fd(). If
    manager_dispatch_notify_fd() fails and returns an error then the handling
    of service notifications will be disabled entirely leading to a
    compromised system. (side issue of LP: #1628687)

 -- Martin Pitt <email address hidden> Tue, 04 Oct 2016 21:43:04 +0200

[Martin Pitt (pitti)]

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.