229 backport for race between explicit mount and handling automount

Bug #1709649 reported by Pegerto Fernandez on 2017-08-09
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Undecided
Unassigned
Xenial
High
Unassigned
Zesty
Medium
Unassigned
Artful
Undecided
Unassigned

Bug Description

[Impact]
In systemd prior to 234 a race exists between .mount and .automount units such that automount requests from kernel may not be serviced by systemd resulting in kernel holding the mountpoint and any processes that try to use said mount will hang. A race like this may lead to denial of service, until mount points are unmounted.

[Testcase]
Create a race between .mount and .automount units, such that automout request is serviced after .mount unit has been started. Observe a hang.
More detailed steps are available at https://github.com/systemd/systemd/pull/5916

[Butfix]
Cherrypick upstream commit https://github.com/systemd/systemd/commit/e7d54bf58789545a9eb0b3964233defa0b007318

[Regression Potential]
The underlying logic of starting/stopping/triggering units is unchanged. However, there the logic as to when to send automout_send_ready() is relaxed, such that it is always sent whenever unit is already mounted. This is done to explicitly cope with late arrival of the incoming [aircraft] automount request.

[Original Bug report / request]

Hi,

We have a blocking issue in systemd for the following release

```
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
```

This release runs systemd229, we are affected by the following auto-mouting race condition

```
https://github.com/systemd/systemd/pull/5916
```

Is back porting the fix to the release 229 an option?

Regards.

CVE References

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in systemd (Ubuntu):
status: New → Confirmed
Dimitri John Ledkov (xnox) wrote :

When you say "we" - who do you mean? Do you per chance have Ubuntu Advantage and/or any other support contracts? If that is the case, please note, you must use UA portal to escalate issues.

Please note the bug report is incomplete w.r.t. infomration of the affected systems. Please use
$ ubuntu-bug systemd
To collect and file complete bug reports.

Nonetheless I can check the mentioned pull requests, and assess if it is applicable for an SRU.

Dimitri John Ledkov (xnox) wrote :

There is no time commitment for this request.

Lars Müller-Gilberger (larsmg) wrote :

Hi Dimitri,

We are affected by that,too.
We have no advantage support.
We have multiple server parks with 3000+ Ubuntu machines.

We sees this this bug triggering especially, if some program/container/pod is using auto mount directly after startup.

We worked around this issue by manually patching, but this is not a great solution, as you can guess.

Is there any expectation when the MR above will be back ported or a new systemd release, which is including the fix will be issued ?

Best Regards
Lars

Changed in systemd (Ubuntu Xenial):
importance: Undecided → High
status: New → Confirmed
Dimitri John Ledkov (xnox) wrote :

Artful has this change already.
Zesty and Xenial need this cherry-pick.

Changed in systemd (Ubuntu Artful):
status: Confirmed → Fix Released
Changed in systemd (Ubuntu Zesty):
status: New → Triaged
Changed in systemd (Ubuntu Xenial):
status: Confirmed → Triaged
Changed in systemd (Ubuntu Zesty):
importance: Undecided → Medium
description: updated

Hello Pegerto, or anyone else affected,

Accepted systemd into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/232-21ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Zesty):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-zesty
Dimitri John Ledkov (xnox) wrote :

Reproduced hang with systemd 232-21ubuntu5.

Upgrading to 232-21ubuntu7 resolves the race.

tags: added: verification-done verification-done-zesty
removed: verification-needed verification-needed-zesty
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 232-21ubuntu7

---------------
systemd (232-21ubuntu7) zesty; urgency=medium

  * networkd: accept `:' in ifnames in systemd/networkd. (LP: #1714933)
  * networkd: add support for ActiveSlave and PrimarySlave netdev options.
    (LP: #1709135)
  * Cherrypick upstream fix for a race between .mount and .automount units,
    which currently may result in automounts hanging. (LP: #1709649)
  * systemd.postinst: Fix-up version number check in the previous sru.
    The version check in the postinst was too tight, thus the SRU fix failed
    validation. (LP: #1710410)

systemd (232-21ubuntu6) zesty; urgency=medium

  * link: Fix offload features initialization.
    This fixes a regression introduced in v232 which caused TCP
    segmentation offloads being disabled by default, resulting in
    significant performance issues under certain conditions. (Closes: #864073)
    (LP: #1703393)
  * loginctl: Fix loginctl ignoring user given session IDs at command-line
    (LP: #1682154)
  * Disable fallback DNS servers.
    This causes resolved to call-home to google, attempt to access network when
    none is available, and spams logs. (LP: #1449001)
  * initramfs-tools: trigger udevadm add actions with subsystems first.
    This updates the initramfs-tools init-top udev script to trigger udevadm
    actions with type specified. This mimicks the
    systemd-udev-trigger.service. Without type specified only devices are
    triggered, but triggering subsystems may also be required and should happen
    before triggering the devices. This is the case for example on s390x with zdev
    generated udev rules. (LP: #1713536)
  * Enable systemd-resolved by default. (LP: #1710410)
  * core: fix systemd failing to serialize tasks correctly on daemon-reload.
    (LP: #1702823)

 -- Dimitri John Ledkov <email address hidden> Wed, 04 Oct 2017 14:22:02 +0100

Changed in systemd (Ubuntu Zesty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 229-4ubuntu21.1

---------------
systemd (229-4ubuntu21.1) xenial-security; urgency=medium

  * SECURITY UPDATE: remote DoS in resolved (LP: #1725351)
    - debian/patches/CVE-2017-15908.patch: fix loop on packets with pseudo
      dns types in src/resolve/resolved-dns-packet.c.
    - CVE-2017-15908
  * SECURITY UPDATE: access to automounted volumes can lock up
    (LP: #1709649)
    - debian/patches/CVE-2018-1049.patch: ack automount requests even when
      already mounted in src/core/automount.c.
    - CVE-2018-1049

 -- Marc Deslauriers <email address hidden> Thu, 01 Feb 2018 07:42:30 -0500

Changed in systemd (Ubuntu Xenial):
status: Triaged → Fix Released

Thanks for porting this to Xenial. Should the Xenial Ubuntu Base release be updated? The Ubuntu Base 16.04.3 does come with the problematic systemd, and on some systems, like the armhf, booting to a bare rootfs is impossible (always get the issue, which manifest itself as 'A start job is running for dev-ttys0.device' message and hang pretty much right after the 1:30 minute delay timeout.

Or, without releasing an Ubuntu Base update, the documentation (Wiki) probably should include a note on this (i.e. 'To document'). qemu could be used to update the systemd package, or possibly downloading and manually updating the newer systemd related files to the extracted rootfs.

Regards,
Eric

Ok, I just tried with this version, and the issue I am seeing sadly does persist. So, this does not resolve it. Please ignore the last comment.

Sorry,
Eric

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers