Ctrl-C message displayed without any actual disk check

[Kees Cook]

[Impact]

 * fsck messages is shown by plymouth, despite all fscks already completed.

[Test Case]

 * Install Ubuntu Desktop with full disk encryption

 * Observe that fsck messages (press ctrl+c) is shown, and remains there on screen for a little while as plymouth transitions to gdm

 * After installing the update, observe that when fsck completes the messages are cleared and the transition to gdm is done without fsck/ctrl+c message present.

 * After this change lands in -updates, and daily desktop images are built with this, double check that casper-md5sum transition is also smooth.

[Regression Potential]

 * plymouth spinner theme is adjusted to clear the footer, at the end of fsck, even if systemd-fsckd didn't do that. At the moment there are two consumers of the fsck apis systemd-fsckd & casper-md5sum used on installed systems and live-isos respectively. If something else pushes messages to the footer, they might get cleared at the end of fsck.

[Other Info]

 * Original bug report

It seems the transition to bgrt lost something with Ubuntu's disk check details. The only thing I see on my screen during a long disk check is the "press Ctrl-C to stop all in progress disk checks" with no progress.

[Daniel van Vugt (vanvugt)]

I've seen the message too, but am not sure there's really a disk check happening at all.

Please run this command to attach more info about the affected machine:

  apport-collect 1880250

[Dimitri John Ledkov (xnox)]

So i think this is what happens:

systemd-fsckd starts, flashes the message on how to skip it.

Realiases there are no fscks to run, and exits, without clearing the Ctrl+C imaage.

Then we freeze that output until gdm starts.

Previously, we only flashed "Ctrl+C message" upon receiving the first update from fsck, and cleared it after fsckd exits. This is not happening anymore.

I'm not sure if fsckd is sending the wrong messages, or if plymouth is not processing them correctly.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dimitri John Ledkov (xnox)]

Plymouth does not clear the footer at the end of fsck.

But with that fix in-place it still looks like fsckd flickers the ctrl+c message, despite not having any progress to show.

[Daniel van Vugt (vanvugt)]

Are you sure about that? Is the footer functionality explicitly bound to fsck or is it more general? I thought (from memory) that it was more general and therefore it's the client's job to clear it. But I'm only going from memory. Glad you fixed it.

[Launchpad Janitor (janitor)]

This bug was fixed in the package plymouth - 0.9.5-0ubuntu2

---------------
plymouth (0.9.5-0ubuntu2) groovy; urgency=medium

  * Clear twostep footer, at the end of fsck, and clear lingering Ctrl+C
    message LP: #1880250

plymouth (0.9.5-0ubuntu1) groovy; urgency=medium

  * New upstream version (lp: #1886886)
  * debian/control:
    - remove the intltool Build-Depends, the new version uses gettext
  * debian/patches/git-drm-improve.patch,
    debian/patches/gitlab-center-messages.patchs:
    - removed, the changes are in the new version
  * debian/libplymouth5.symbols:
    - refreshed for the new version
  * debian/patches/gitlab_correct_keynames.patch:
    - use the correct title and subtitle key names

 -- Dimitri John Ledkov <email address hidden> Wed, 12 Aug 2020 22:03:42 +0100

[Dan Streetman (ddstreet)]

@rbalint, @didrocks, @mbiebl, we've been carrying the systemd-fsckd custom patch for 5 years now (via Debian)...can we either get it upstreamed, or drop it? I don't think continuing to carry it ourselves is a good idea. Note that not only is there this bug, but also the systemd-fsckd autopkgtest has been regularly failing, per bug 1892358.

[Michael Biebl (mbiebl)]

I would be happy to see this patch merged upstream.
There was an attempt a few years ago:
https://lists.freedesktop.org/archives/systemd-devel/2015-March/029186.html

Eventually it was removed again:
https://github.com/systemd/systemd/commit/96d9117ad2db7d8c13f7898127eee8939e88daf1
https://lists.freedesktop.org/archives/systemd-devel/2015-April/030175.html

[Didier Roche-Tolomelli (didrocks)]

This patch was worked with upstream and even merged in systemd master.
Howwever, after a month, Lennart decided to revert it because he strongly thinks that no feedback on fsck is needed because "everyone have fast SSD disks". (frustrating experience after working for a good 3 months getting upstream agreeing on the design and doing review first). Martin Pitt tried to push it again but there was no way to discuss apparently.

We decided that it wasn’t acceptable 5 years ago, especially for desktop and server users, to sit in front on a black screen for minutes in case of spinning disks because a fsck is happening without any feedback that things are progressing (with the risk of the user forcibly shutting down his machine, thinking the boot is stuck). Has anything changed in that regard?

[Didier Roche-Tolomelli (didrocks)]

Note: from what I see above, the issue is on the plymouth new integration (I think it’s xnox that changed it) modifying the logic displaying the cancelling message. Maybe worth digging there?

[Michael Biebl (mbiebl)]

From my personal POV, I consider the progress bar more important then the ability to interrupt fsck with CTRL+C.
Nowadays, SSDs are much more common and enable_periodic_fsck = 0 has been the default for many years now. So this is much less of an issue today then it was back in 2015, I think.

I think text-only boot (without plymouth and without quiet) has a reasonable enough progress report from fsck. Without fsckd, we'd have no progress report in plymouth and quiet text-only boots.

[Michael Biebl (mbiebl)]

As for text-only boot: Maybe we could convince upstream to enable verbose mode once an fsck takes longer then say 30s (similar to how the eye-of-cyclon animation kicks in when services take a long time to start). With something like that implemented, dropping fsckd might be an option, I think.

[Michael Biebl (mbiebl)]

Turns out, the missing fsck progress bar is a regression:
https://github.com/systemd/systemd/issues/17157

[Brian Murray (brian-murray)]

Hello Kees, or anyone else affected,

Accepted plymouth into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/plymouth/0.9.4git20200323-0ubuntu6.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted plymouth (0.9.4git20200323-0ubuntu6.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

systemd/245.4-4ubuntu3.2 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#plymouth

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Dimitri John Ledkov (xnox)]

installed focal daily with LUKS and plymouth 0.9.4git20200323-0ubuntu6.1 ; i could see that on boot the ctrl+c message is there after unlock, briefly, and then cleared.

Then i downgraded to 0.9.4git20200323-0ubuntu6 and instead of brief ctrc+c message, it lingers for a lot longer until gdm3 starts.

Thus the plymouth portion of the bug is verified.

[Launchpad Janitor (janitor)]

This bug was fixed in the package plymouth - 0.9.4git20200323-0ubuntu6.1

---------------
plymouth (0.9.4git20200323-0ubuntu6.1) focal; urgency=medium

  * Clear twostep footer, at the end of fsck, and clear lingering Ctrl+C
    message LP: #1880250

 -- Dimitri John Ledkov <email address hidden> Wed, 30 Sep 2020 15:01:26 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for plymouth has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Dimitri John Ledkov (xnox)]

Installed focal Ubuntu desktop .2 media with LUKS full disk encryption.

On boot the fsck messages briefly flicker, and then are cleared, then there is smooth transition to gdm.

It's a lot better than before, where fsck message would linger forever.

It would be ideal to make it flickerfree. Aka delay displaying fsck message if it takes more than 1second and/or there is an update.

[Dimitri John Ledkov (xnox)]

Request for truely flicker free boot is in https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1914409