fsck not running at all on reboot

Bug #1797282 reported by tomtom
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

After upgrading some servers from 16.04 to 18.04 I'm met with a MOTD that says:

*** /dev/xvda1 should be checked for errors ***

I added "fsck.mode=force" to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub before running 'sudo update-grub'. I also verified that fsck was present in /boot/grub/grub.cfg afterwards, before rebooting the server.

Surprisingly I was met with the same error message in MOTD when logging in. I then ran tune2fs to see when the last check was performed and this is the output:

$ sudo tune2fs -l /dev/xvda1 | grep checked
Last checked: Wed Sep 12 16:17:00 2018

I then tried to change mount count to 1 with tune2fs -c 1 /dev/xvda1, but after another reboot I was still met with the same error message and the timestamp for last checked was unchanged.

I have the same problem on all the servers that was upgraded from 16.04 to 18.04, but also on a new server installed directly with 18.04.

In should be mentioned that the servers are AWS EC2 instances, so I have no way of trying to run fsck from a liveusb.

Another user has reported the same issue here:
https://ubuntuforums.org/showthread.php?t=2403368

I'm not quite sure what information is needed, but attached are some basic information about the system.

Please let me know if you need any any other outputs or logs.

Edit1: I tested fsck.mode=force on my laptop running Ubuntu 18.04.1 LTS (Xubuntu) and it works fine. Seems to be related to the server edition.

Tags: bionic
Revision history for this message
tomtom (tbjornli) wrote :
tomtom (tbjornli)
summary: - fsck not running on reboot when fsck.mode=force is set
+ Ubuntu 18.04 Server: fsck not running at all on reboot
summary: - Ubuntu 18.04 Server: fsck not running at all on reboot
+ fsck not running at all on reboot
description: updated
Revision history for this message
dman777 (coolio) wrote :

I am having this issue also.

fsck is not running on my root drive on Ubuntu 18.04 LTS server edition
I have tried fsck.mode=force in grub on the kernel command line to no success.
I have set the mount count to 1 with tune2fs -c 1 /dev/sda2 and rebooted and still no fsck. Instead I just get a text message saying you need to do a fsck on /dev/sda2.

--------------------------------------------------

one@localhost:~$ sudo tune2fs -l /dev/sda2

tune2fs 1.44.1 (24-Mar-2018)
Filesystem volume name: <none>
Last mounted on: /
Filesystem UUID: 36589912-c6ab-11e8-93a6-7085c27cfaec
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 6160384
Block count: 24641536
Reserved block count: 1232076
Free blocks: 19441110
Free inodes: 5417720
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64
Reserved GDT blocks: 1024
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Flex block group size: 16
Filesystem created: Tue Oct 2 20:25:28 2018
Last mount time: Tue Oct 9 22:14:49 2018
Last write time: Tue Oct 9 22:13:35 2018
Mount count: 25
Maximum mount count: 1
Last checked: Tue Oct 2 20:25:28 2018
Check interval: 0 (<none>)
Lifetime writes: 179 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
First orphan inode: 1591485
Default directory hash: half_md4
Directory Hash Seed: 5dcfe344-2b26-4b50-97e1-21952c171d2b
Journal backup: inode blocks
Checksum type: crc32c Checksum: 0x305a8ddb

I am not running on cloud, just a home PC with AMD Ryzen 5

-------------------------------------------

Linux localhost 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic

Paul White (paulw2u)
affects: ubuntu → e2fsprogs (Ubuntu)
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in e2fsprogs (Ubuntu):
status: New → Confirmed
Revision history for this message
dman777 (coolio) wrote :

Can this be bumped up to a high importance? This is a major issue as not checking the file system leads to a broken system.

tags: added: bionic
Revision history for this message
Theodore Ts'o (tytso) wrote :

This is probably not assigned to the right package. It's not an issue with e2fsprogs, but whatever component is running fsck. This might be systemd, or upstart, or whatever the heck Ubuntu is using these days. I lost track a while ago. :-)

I can tell, you as the Debian maintainer of e2fsprogs, that fsck runs *just* *fine* on Debian testing. With Debian, using systemd (which is the default), the systemd units which run fsck are generated by the systemd-fstab-generator program which is part of systemd.

So the main issue is whether anyone on the Canonical/Ubuntu team is paying attention. I pay attention in case people report e2fsprogs bugs, but this is not an e2fsprogs issue. And I don't use Ubuntu so I can't really help.

Revision history for this message
tomtom (tbjornli) wrote :

Changed package based on comment made by Theodore Ts'o

affects: e2fsprogs (Ubuntu) → systemd (Ubuntu)
Revision history for this message
Brian Murray (brian-murray) wrote :

I've been unable to recreate this on an Ubuntu installation of Ubuntu 16.04.5 which I then upgraded to Ubuntu 18.04. If I set 'tune2fs -c1' for my root partition it is checked for errors on every boot. I'll have someone test with an AWS EC2 instance.

@dman777 while you say you set -c1 the output of tune2fs indicates it (Check interval) is set to 0.

Changed in systemd (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Phil Pemberton (philpem) wrote :

I'm seeing this on an Ubuntu 16.04.5 LTS system running on top of LVM. In my case, I get:

*** /dev/mapper/wolf-root will be checked for errors at next reboot ***

Having to dig out a 16.04 LTS Server USB stick every time I need to run FSCK is extremely inconvenient. It turns an unattended maintenance job into one that requires a lot of hand-holding.

Please let me know if there's any information I can provide to get this fixed.

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for systemd (Ubuntu) because there has been no activity for 60 days.]

Changed in systemd (Ubuntu):
status: Incomplete → Expired
Ewan Mellor (ewanmellor)
Changed in systemd (Ubuntu):
status: Expired → Confirmed
Revision history for this message
Ewan Mellor (ewanmellor) wrote :

In my case, Ubuntu 18.04 running on AWS, the problem was as follows:

1. /etc/fstab had 0 in the 6th (passno) field. I presume this is the default that AWS places there, or maybe this is determined by the Ubuntu template on AWS.
2. According to a comment that I found on Stack Overflow, this is used by mkinitramfs to decide not to put fsck in the initrd. This is *in no way* obvious.
3. Without fsck in the initrd, it can't run the fsck, so it just prints a message instead. This message tells you that a fsck is recommended, but it doesn't tell you why it hasn't done one.

To fix this:
1. I changed /etc/fstab to have a "1" in the passno field
2. update-initramfs -u
3. reboot, and the check runs as expected.

I think the following changes should be made:

1. There should be a console warning during startup if the root volume cannot be checked because fsck is not available.
2. If /etc/fstab is coming from an Ubuntu-controlled template, then passno for the root volume should be set to 1. I know that AWS will fail to boot the instance if the fsck goes interactive, necessitating manual repairs, but surely failing to boot the instance is better than silently running on a corrupt root filesystem.
3. This should be in some documentation somewhere. The fact that mkinitramfs will exclude fsck from the initrd if it thinks you don't need it is particularly obscure. If there is some kind of "getting started with Ubuntu on AWS" then this should be in there.

Revision history for this message
Dan Streetman (ddstreet) wrote :

please reopen if this is still an issue

Changed in systemd (Ubuntu):
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.