systemd: handle undelegated cgroup2 hierarchy

[Christian Brauner]

[Impact]

 * When a container is presented with a unified cgroup hierarchy, which is not properly delegated, systemd should not attempt (and fail) to use. This improves compatibility of xenial containers running on unified cgroup hierarchy hosts.

[Test Case]

 * Xenial containers should boot, with non-writable unified cgroup hierarchy hosts.

[Regression Potential]

 * unified cgroup hierarchy is not in use by default on xenial hosts, thus this is forward compatibility improvment with e.g. bionic hosts running xenial containers.

[Other Info]

 * Original bug report

Hey everyone,

Current systemd versions all fail when the unified cgroup hierarchy is not-writable. This is especially problematic in containers where the systemd administrator might decide to not delegate the unified hierarchy or when running with a liblxc driver that doesn't yet know how to handle the unified cgroup hierarchy. I've pushed patches to systemd upstream that let systemd ingnore the non-delegated unified hierarchy. The relevant commits are:

e07aefbd675b651f8d45b5fb458f2747b04d6e04
2d56b80a1855836abf1d7458394c345ad9d55382
1ff654e28b7b8e7d0a0be33522a84069ac6b07c0

These patches will be in 236 but should be backported from xenial upwards.

Christian

[Dimitri John Ledkov (xnox)]

These did not make v235, I guess you meant v236. Description updated.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 235-3ubuntu3

---------------
systemd (235-3ubuntu3) bionic; urgency=medium

  * netwokrd: add support for RequiredForOnline stanza. (LP: #1737570)
  * resolved.service: set DefaultDependencies=no (LP: #1734167)
  * systemd.postinst: enable persistent journal. (LP: #1618188)
  * core: add support for non-writable unified cgroup hierarchy for container support.
    (LP: #1734410)

 -- Dimitri John Ledkov <email address hidden> Tue, 12 Dec 2017 13:25:32 +0000

[Dimitri John Ledkov (xnox)]

I'm slightly perplexed if this is needed in xenial or not.

[Brian Murray (brian-murray)]

Hello Christian, or anyone else affected,

Accepted systemd into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/234-2ubuntu12.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dimitri John Ledkov (xnox)]

@cbrauner

Could you please verify this bug report for me, and state the versions of systemd used on the host / container / subcontainter?

Otherwise, I might be forced to drop this patch.

[Łukasz Zemczak (sil2100)]

Next time, please provide some information about what and how was tested when setting verification-done for an update. It is not enough to just set the tag. The SRU team needs to be made aware of what has been tested and how, if the test case was followed as expected. The more verbose the better.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 234-2ubuntu12.3

---------------
systemd (234-2ubuntu12.3) artful; urgency=medium

  [ Dimitri John Ledkov ]
  * Fix test-functions failing with Ubuntu units. LP: #1750608
  * tests: switch to using ext4 by default, instead of ext3. LP: #1750608
  * Fix kdump service not starting, due to systemd not loading dropins.
    Cherrypick a fix from upstream. (LP: #1708409)
  * systemd-fsckd: Fix ADT tests to work on s390x too. (LP: #1736955)
  * netwokrd: add support for RequiredForOnline stanza. (LP: #1737570)
  * resolved.service: set DefaultDependencies=no (LP: #1734167)
  * systemd.postinst: enable persistent journal. (LP: #1618188)
  * core: add support for non-writable unified cgroup hierarchy for container support.
    Rebase and de-fuzz. (LP: #1734410)
  * Prevent MemoryDenyWriteExecution policy bypass, by disallowing pkey_mprotect when mprotect is disallowed.
    CVE-2017-15908 (LP: #1725348)
  * networkd: enable promote_secondaries on networkd managed dhcp links.
    This fixes failing to renew DHCP lease, on networkd managed devices.
    (LP: #1721223)

  [ Kleber Sacilotto de Souza ]
  * systemd-rfkill service times out when a new rfkill device is added
    - rfkill-fix-erroneous-behavior-when-polling-the-udev-.patch: Comparing
    udev_device_get_sysname(device) and sysname will always return true. We need to
    check the device received from udev monitor instead.
    - rfkill-fix-typo.patch: Fix typo in rfkill log message. (LP: #1734908)

 -- Dimitri John Ledkov <email address hidden> Tue, 20 Feb 2018 16:11:58 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Christian Brauner (cbrauner)]

Sorry for the brevity before. I tested this with systemd 23{5,6}
inside xenial and artful containers which is really the only case
where it matters.

A systemd with my patch applied would happily:
1. skip over undelegated /sys/fs/cgroup/unified mountpoints
   (e07aefbd675b651f8d45b5fb458f2747b04d6e04).
2. skip over undelegated pur cgroup2 mountpoints at /sys/fs/cgroup
   (2d56b80a1855836abf1d7458394c345ad9d55382)
3. remove any empty mountpoints created for case 1. and 2.
   (1ff654e28b7b8e7d0a0be33522a84069ac6b07c0)

Thanks for backporting these patches!
Christian

[Dimitri John Ledkov (xnox)]

I'm confused about this backport to xenial, given that xenial's mount-setup.c does not try to neither mount nor use /sys/fs/cgroup/unified.

It wants to instead use /sys/fs/cgroup (for unified case).

And it has no capability to use hybrid cgroups.

Is this backport really needed back to xenial? I fear it may then require backporting cg_is_hybrid_wanted aka hybrid cgroup support as well.

[Christian Brauner (cbrauner)]

If the systemd version doesn't support hybrid cgroup layout on xenial then fine but I thought it did. But please make sure that Xenial doesn't have anything mounted on /sys/fs/cgroup/unified.

[Stéphane Graber (stgraber)]

There is no /sys/fs/cgroup/unified on xenial, closing.