filesystems are never fsck'ed if they live on removable USB disks

[Mario Vukelic]

I don't know which package to assign this bug to, and sorry if it has been reported before - I couldn't find a report.

ext2/3 filesystems are regulary fsck'ed after a given number of boots if those filesystems are part of the fixed disks. However removable USB disks (I can only check for those, but I guess firewire at al. also share this) are never fsck'ed automatically. This might me dangerous because they are much more likely to be removed incorrectly (w/o unmounting) and are attached/reattached frequently. I just realized that my external disk was mounted hundreds of times without ever being checked (and am sure that despite trying to be responsible, I sometimes yanked the cable w/o unmounting)

NOTE:
i've removed the ext3 case because this also applies to some other filesystem types as described in duplicated bugs.

[PeteDonnell (pete-donnell-deactivatedaccount)]

I'd like to add a further comment to this bug. I have a computer running Kubuntu Feisty with an external hard drive which is formatted with JFS. At some point it was apparently removed from the computer without being properly unmounted (I'm not sure if this was due to a system crash or someone being careless as I'm not the only user of the computer, and it happened when someone else was using it). The next time the external drive was connected, as usual the KDE automount popup appeared, but when a user attempted to access the disk it failed to mount. I'm afraid I don't remember the exact error message it gave, but it wasn't too helpful. I tried mounting the drive by hand, and that didn't work either. I checked 'dmesg | tail' to see if any information appeared there, and there was a message about JFS. I'm pretty sure that it didn't mention anything about the drive being improperly unmounted, but I guessed that this was the cause of the problem and ran fsck. Sure enough, once fsck had run the drive mounted just fine.

The point I'm trying to make here is that while I've been using Linux for a few years and didn't have too much trouble figuring out what the problem was, the other users of the computer were stumped. The fact that external drives are never fsck'ed is definitely a bad thing from the point of view of data integrity, but if the automounter doesn't detect when an external drive hasn't been properly unmounted and offer the option of checking the system most users are going to be much more confused when the drive won't mount, which I think is a more serious problem.

I realise this isn't strictly an Ubuntu bug but exists upstream too.

[Mario Vukelic (kreuzsakra)]

I'd like to add that the output of mke2fs contains, "this filesystem will be automatically checked every 23 mounts or 180 days, whichever comes first", which is just plain not correct and gives a false sense of security.

[Andres Mujica (andres.mujica)]

Hi, thanks for your Bug Report, it seems to be a feature enhancement, but in order to complete the report can you reproduce the problem and debug it using this procedure at

https://wiki.ubuntu.com/DebuggingRemovableDevices

With this we would have enough information for a developer to take it.

Also it could be a good idea to write or look for a spec or blueprint that proposes this feature.

[Mario Vukelic (kreuzsakra)]

Well, I'm not sure what's there to reproduce or debug.
In general, nobody seems to be sure whether ext3 actually needs those checks. Recently there was a long discussion about the regular forced checks of fixed disks on the ubuntu-devel-discuss list, and it was inconclusive [1]. I suggested that Ubuntu should talk to an ext3 developer, but seemed to get no reply. I wonder whether writing a spec or blueprint is of any help right now, if the fundamental question is not answered even for fixed disks.

[1] Starts here: https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2007-December/002744.html

[Mario Vukelic (kreuzsakra)]

Actually there is something conclusive in this discussion: https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2007-December/002764.html

"To try to answer the question of whether we could simply disable the
periodic fsck, I decided to ask Mingming Cao, one of the developers
who has worked on ext3 and later, ext4. I just got the following:

"Periodically fsck ext3 is still needed, even if ext3 is a journalled fs.
kernel code vm/fs could be buggy, or disks IO errors, which cause
filesystem metadata corrupted silently, this can't be detected by simply
replaying the journal log.

[....]"

That must go doubly for removable disks, due to more likely unclean removal and additional risk of hardware issues (disks being carried around, etc.)

And here Martin Pitt mentions a UDS decision about how to deal with fsck for fixed disks: https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2007-December/002753.html

[Andres Mujica (andres.mujica)]

i wonder if the right spec to enhances with this wishlist is this one

https://wiki.ubuntu.com/DesktopTeam/Specs/PartitionManagement

I suggest to check it, find another one more suitable for the purpose or even - if it's possible - to create one.

[Tommy Trussell (tommy-trussell)]

I have been using an external drive formatted with ext3 for about a year through several Ubuntu releases, and because I shut down that system almost every night, I find that about once a month the drive won't mount and I have to fsck it before it mounts normally -- you can see the "too many mounts" message if you look in the logs but unfortunately there's NO indication in the GUI of the source of the problem.

[loke (developer-loke)]

Seems like no one cares about a healthy file system. It is a critical issue, especially if Gnome has to support automount, because file systems are never checked. Running a simple fsck, and asking the user to wait, or to even show a terminal should not be a problem.

[Tommy Trussell (tommy-trussell)]

@loke: surely everyone cares about a healthy file system, but the problem is thorny. I see several blueprints to handle "ordinary" filesystem fsck -- such as moving the fsck to shutdown, disabling fsck on battery power, asking user if they want to schedule it, etc. None of the ones I found seems to include a use case involving external drives.

@Andres Mujica: I just looked at the status of the PartitionManagement spec and blueprint, and it has been marked as "superseded." Its use cases also didn't include fsck on an external drive, so I think another blueprint and spec needs to be created to handle this use case.

I just did a few searches on blueprints, and there was a lot of work (in the Gutsy era) on handling a full filesystem situation, though from my reading this apparently involved a full root filesystem. It was also focused on making the system usable enough to recover from the full filesystem situation on boot -- not a full filesystem being mounted on an already active system.

What does modern Gnome do or say when you fill an external USB drive? Does it put up a warning bubble? I believe it doesn't but I haven't tried yet in Karmic. A process that watches for a full filesystem might also watch for other important notices regarding filesystems. I think someone proposed some dbus messages getting created to handle filesystem fsck situations, and that would definitely be a place to look at.

I have an old PowerBook running Ubuntu 6.06.1 Dapper PowerPC running XFCE and it pops up a warning whenever I mount a filesystem having less than 10% free space. I haven't tried a damaged system that needs a fsck. BUT I believe the full warning is an option in the file manager, or maybe I activated a daemon for that. I don't remember -- I installed that system a long time ago. ;-)

[Mario Vukelic (kreuzsakra)]

In Natty:
When you log in at the console with removable disks attached which require an fsck (due to being mounted too often without one) , bash tells you that they will be checked on the next reboot, which is again plain untrue and gives you a false sense of security.

[David Clayton (dcstar)]

C,mon developers, this is another one of these EMBARRASSING things that gives Linux a bad name.

All removable devices should be auto fsck'd as a strict policy to ensure that the user has the best chance of accessing and saving their data. At least give people a warning and the option to fix things.

[Mario Vukelic (kreuzsakra)]

Recently I had to support an Ubuntu-using friend whose external FAT disk was not mounting because it had previously been unplugged without unmounting. I told him repeatedly about the need to unmount, but sometimes he still forgets or it just happens by accident. This is a casual user who does not know about fsck; even if he did he would probably be overwhelmed by the process. Had he not asked me, he would have considered the disk dead and the data lost.

[Phillip Susi (psusi)]

Automatic periodic fscks have long annoyed many people and don't do any good, which is why we have stopped doing them on internal disks. It would likely be much more of an annoyance on removable media. Ext3/4 don't need such checks even if you unplug them without unmounting. For the filesystems that do ( fat, ntfs ), we don't have a way of detecting when they need checked, or decent tools to properly check them anyhow.