"system is booting up" while trying to log in after installing snapd on ubuntu 14.04

[Zygmunt Krynicki]

[Impact]

 * Deputy systemd sometimes wrongly prevents TTY logins
 * Since it is deputy systemd, it should not be controlling PAM stack nologin feature, i.e. deputy systemd should not create /run/nologin since it's not removing it reliably.

[Test Case]

 * On boot, /run/nologin should not exist
 * Logins should not be prevented

[Regression Potential]

 * This impacts systems with snapd and deputy systemd only (not the default configuration)

[Original Descriptions]

I've installed snapd on an up-to-date Ubuntu 14.04 server system running LTS enablement kernel.

After toying with snapd and some simple snaps I logged out (so far everything was OK).
I returned to the console after a while (I'm not sure if I logged out but I suspect I had to) and I saw the login prompt. After entering my username a line was printed "System is booting up" and I was kicked back ot the login prompt (It never asked for my password).

After rebooting the problem went away.

[Thomas Voß (thomas-voss)]

This bug report against systemd seems relevant:
  https://github.com/systemd/systemd/issues/3436

Other distributions are experiencing the issue, too:
  https://bugzilla.opensuse.org/show_bug.cgi?id=980324

[Thomas Voß (thomas-voss)]

I can occasionally reproduce the issue with an autopkgtest setup using a qemu vm. Local spread tests using the qemu backend work fine, though.

[Steve Langasek (vorlon)]

what version of the systemd package do you have installed?

[Paul Collins (pjdc)]

With a freshly-booted up to date trusty VM (ubuntu-trusty-daily-amd64-server-20170130-disk1.img, then dist-upgraded for libssl1.0.0 openssl), simply installing systemd seems to be sufficient to cause the file to exist.

ubuntu@pjdc-test:~$ ls -l /var/run/nologin
ls: cannot access /var/run/nologin: No such file or directory
ubuntu@pjdc-test:~$ sudo apt-get install systemd
sudo: unable to resolve host pjdc-test
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  libsystemd-journal0
Suggested packages:
  systemd-ui
The following NEW packages will be installed:
  libsystemd-journal0 systemd
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,476 kB of archives.
After this operation, 9,229 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu/ trusty-updates/main libsystemd-journal0 amd64 204-5ubuntu20.22 [51.0 kB]
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu/ trusty-updates/main systemd amd64 204-5ubuntu20.22 [1,425 kB]
Fetched 1,476 kB in 0s (17.6 MB/s)
Selecting previously unselected package libsystemd-journal0:amd64.
(Reading database ... 51302 files and directories currently installed.)
Preparing to unpack .../libsystemd-journal0_204-5ubuntu20.22_amd64.deb ...
Unpacking libsystemd-journal0:amd64 (204-5ubuntu20.22) ...
Selecting previously unselected package systemd.
Preparing to unpack .../systemd_204-5ubuntu20.22_amd64.deb ...
Unpacking systemd (204-5ubuntu20.22) ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up libsystemd-journal0:amd64 (204-5ubuntu20.22) ...
Setting up systemd (204-5ubuntu20.22) ...
Initializing machine ID from D-Bus machine ID.
systemd start/running, process 3110
Processing triggers for libc-bin (2.19-0ubuntu6.9) ...
Processing triggers for ureadahead (0.100.0-16) ...
ubuntu@pjdc-test:~$ ls -l /var/run/nologin
-rw-r--r-- 1 root root 21 Feb 1 01:31 /var/run/nologin
ubuntu@pjdc-test:~$

[Paul Collins (pjdc)]

I forgot to mention that pam_nologin prints "System is booting up." and fails non-root logins when /var/run/nologin exists, which is why that is relevant. :)

[Thomas Voß (thomas-voss)]

I did some digging into systemd-tmpfiles and its configuration. In the trusty case, with systemd *not* running as PID 1, the systemd-tmpfiles does not strictly need to create /run/nologin to shield state transitions. For that, I removed creation of the file from trusty's systemd-tmpfiles configuration. Packages for testing purposes are in ppa:thomas-voss/trusty.

@Paul: Would be great if you could give the package a round of testing, too.

[Thomas Voß (thomas-voss)]

I managed to reproduce the issue without snapd being involved at all. Marking as invalid for snapd, bumping priority to "high". Among other things, this blocks autopkgtests for snapd on trusty.

[Zygmunt Krynicki (zyga)]

I've installed the updated systemd from tvoss' PPA and I didn't see anything wrong anymore.

[Paul Collins (pjdc)]

Also no longer able to reproduce when installing systemd from tvoss's PPA on a fresh VM (ubuntu-trusty-daily-amd64-server-20170131-disk1.img).

[Andy Whitcroft (apw)]

Hello Zygmunt, or anyone else affected,

Accepted systemd into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/204-5ubuntu20.24 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dimitri John Ledkov (xnox)]

installing systemd 204-5ubuntu20.22, on pure trusty machine does result in /run/nologin file being created. (WRONG)

installing systemd 204-5ubuntu20.24 does not result in /run/nologin file being created. (GREAT)

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 204-5ubuntu20.24

---------------
systemd (204-5ubuntu20.24) trusty; urgency=medium

  [ Thomas Voß ]
  * Do not create /run/nologin, and thus make sure deputy systemd does not
    prevent system logins. LP: #1660573.

systemd (204-5ubuntu20.23) trusty; urgency=medium

  * d/p/0001-libudev-util-change-util_replace_whitespace-to-retur.patch,
    d/p/0002-udev-event-add-replace_whitespace-param-to-udev_even.patch,
    d/p/0003-udev-rules-perform-whitespace-replacement-for-symlin.patch:
    Cherry-pick upstream fixes from Dan Streetman <email address hidden> to
    fix by-id symlinks for devices whose IDs contain whitespace.
    LP: #1647485.

 -- Dimitri John Ledkov <email address hidden> Thu, 02 Feb 2017 10:29:19 +0000

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.