console-setup failure due to race with systemd-tmpfiles
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
console-setup (Ubuntu) |
Fix Released
|
High
|
Steve Langasek | ||
Xenial |
Fix Released
|
High
|
Steve Langasek | ||
Bionic |
Fix Released
|
High
|
Steve Langasek | ||
Cosmic |
Fix Released
|
High
|
Steve Langasek | ||
Disco |
Fix Released
|
High
|
Steve Langasek |
Bug Description
[SRU Justification]
I'm seeing a console-
where the temp file that should have been created can't be found.
This is a regular xenial cloud image.
19:51:13 systemd-
...
19:51:13 systemd[1]: Starting Set console font and keymap...
19:51:15 setupcon[455]: /bin/setupcon: 809: /bin/setupcon: cannot open /tmp/tmpkbd.a8FGSs: No such file
19:51:15 systemd[1]: console-
19:51:15 systemd[1]: Failed to start Set console font and keymap.
19:51:15 systemd[1]: console-
19:51:15 systemd[1]: console-
...
/bin/setupcon has a lovely tempfile function that looks like:
if \
TMPFILE=`mktemp /tmp/tmpkbd.XXXXXX 2>/dev/null` \
|| TMPFILE=`mktemp /run/tmpkbd.XXXXXX 2>/dev/null` \
|| TMPFILE=`mktemp /dev/.tmpkbd.XXXXXX 2>/dev/null` \
|| TMPFILE=`mktemp /lib/init/
|| TMPFILE=`mktemp 2>/dev/null`
Here's our edited IRC conversation on the bug:
<@vorlon> so I do think you're being hit by the tmp cleaner
<@vorlon> also this seems like bad pathological default behavior for
the tmp cleaner, to delete files that have just been created
<@vorlon> but we should fix console-setup to not rely on /tmp
<@vorlon> and I prefer that we do that instead of trying to fiddle with
the ordering of the systemd units on startup
<@vorlon> i.e. console-setup has an undeclared dependency
on systemd-
instead of declaring it
<@vorlon> are you failing the race more often now than in the past?
<@rcj> vorlon: it feels like it's failing more often but I don't have
data to answer that.
<@vorlon> are we shipping an image with a dirty rootfs?
<@vorlon> dirty in the sense that e2fsck doesn't take one look at it,
say "yep, nothing to do here" and exit
<@vorlon> in the sense that this is what would make dev-sda1.device slow
to complete AIUI
<@rcj> would filesystem resize on first boot mark it dirty? Because
that will happen
<@vorlon> huh good question
<@vorlon> it might
<xnox> rcj, unclean shutdown?
<@rcj> xnox: first boot
[Test case]
1. Install console-setup from -proposed
2. Reboot
3. Verify that `systemctl status console-setup` shows that the service has completed successfully.
Since this is a race, additional fuzz testing may be warranted for the cloud images to confirm that the issue experienced in GCE is really fixed. However, that should not block promotion of this SRU fix since there definitely is a race here that should be fixed per se even if there are other issues still causing a failure in GCE.
[Regression potential]
None known. /run is guaranteed to be mounted rw very early in boot - generally before /tmp is mounted, due to /tmp being on the rootfs that needs to be fscked before remount.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: console-setup 1.108ubuntu15.4
ProcVersionSign
Uname: Linux 4.15.0-1029-gcp x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
Date: Wed Apr 10 19:24:12 2019
PackageArchitec
ProcEnviron:
TERM=xterm-
PATH=(custom, no user)
XDG_RUNTIME_
LANG=en_US.UTF-8
SHELL=/bin/bash
SourcePackage: console-setup
UpgradeStatus: No upgrade log present (probably fresh install)
Changed in console-setup (Ubuntu): | |
importance: | Undecided → High |
status: | New → Triaged |
Changed in console-setup (Ubuntu Bionic): | |
status: | New → Triaged |
importance: | Undecided → High |
Changed in console-setup (Ubuntu Cosmic): | |
status: | New → Triaged |
importance: | Undecided → High |
Changed in console-setup (Ubuntu Disco): | |
status: | Triaged → Fix Committed |
assignee: | nobody → Steve Langasek (vorlon) |
description: | updated |
description: | updated |
description: | updated |
Changed in console-setup (Ubuntu Cosmic): | |
status: | Triaged → In Progress |
Changed in console-setup (Ubuntu Bionic): | |
status: | Triaged → In Progress |
Changed in console-setup (Ubuntu Xenial): | |
status: | New → In Progress |
Changed in console-setup (Ubuntu Cosmic): | |
assignee: | nobody → Steve Langasek (vorlon) |
Changed in console-setup (Ubuntu Bionic): | |
assignee: | nobody → Steve Langasek (vorlon) |
Changed in console-setup (Ubuntu Xenial): | |
assignee: | nobody → Steve Langasek (vorlon) |
importance: | Undecided → High |
tags: | added: id-5caba54405de185b64adc2db |
tags: | added: id-5cae47869ef68677a65c1e71 |
Hello Robert, or anyone else affected,
Accepted console-setup into cosmic-proposed. The package will build now and be available at https:/ /launchpad. net/ubuntu/ +source/ console- setup/1. 178ubuntu9. 1 in a few hours, and then in the -proposed repository.
Please help us by testing this new package. See https:/ /wiki.ubuntu. com/Testing/ EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification- needed- cosmic to verification- done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed- cosmic. In either case, without details of your testing we will not be able to proceed.
Further information regarding the verification process can be found at https:/ /wiki.ubuntu. com/QATeam/ PerformingSRUVe rification . Thank you in advance for helping!
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.