Dosfsck Run On Every Boot on FAT

[Lukas Sabota]

Whenever I book into my Ubuntu system, dosfsck is run on my FAT32 windows partition. This step makes my boot sequence very long. Is this fsck on every boot really neccessary? Perhaps it should be run after a certain amount of mounts or a certain number of days.

This occurs in both Dapper and Edgy.

[Lukas Sabota (punkrockguy318)]

Attaching fstab file

[Carthik Sharma (carthik)]

I have the same issue on my Dapper installation.
This prolongs the time taken to boot. The system has been running Ubuntu since Warty and only now do I have to wait for dosfsck to finish each time when I boot. I usually use a Ctrl+c Ctrl+d to kill the dosfsck if I want to boot faster.

Please advice on whether this is really neccessary, and why we would possibly need this on every boot.

[Daniel Werner (demitsu)]

Agreed. On my system, those dosfscks actually take longer than the remaining boot process combined.

> Perhaps it should be run after a certain amount of mounts or a certain number of days.

The problem is that unlike ext2, Reiser3 et al, FAT doesn't store such when-to-check information in its superblock equivalent, not even when the file system has last been mounted. The latter is kept track of somewhere in the Windows registry instead.

In case the FAT partition in question is not the one where Windows is installed, I see no reasonable way to get at this registry information. In case the FAT partition is actually the one containing the registry, it would have to be mounted first, which at the time it is checked during boot, it hasn't. Parsing the registry is not a trivial task and, in my opinion, not worth the hassle anyway.

The only solution I see (short of storing it on another file system) would be to keep the neccessary information in some part of the FAT filesystem which is generally ignored by Windows and its scandisk/defrag tools, so as not to break Windows compatibility.

Perhaps a reserved sector after the boot sector could be used, provided that doesn't need re-creating of the file system. Could anyone with practical FAT experience please evaluate this? I'm willing to write up a spec if enough people are interested in this feature.

[Duncan Lithgow (duncan-lithgow)]

I appreciate that Daniel would like this to be solved in a similar way to on other file systems. But since fat doesn't support it I suggest we don't try and make it make complicated than necessary, I have these suggested solutions:

* Write a simple hidden text file with configuration info onto the fat partition. I might be called .mount-conf . That file could then hold whatever information is needed. Of course it should have a simple explanation at the start of what it is. And if it's missing or writable the system should perform a default operation.
* The *nix system keeps a record of uptime anyway. The boot sequence could probably be configured to run dosfsck at certain intervals of uptime. I don't know what this would mean for systems which were always-on.

I think that whatever we do it should be configurable in the fstab file.

[Duncan Lithgow (duncan-lithgow)]

Just a note for anyone tring to get around this. If your vfat system is not important to your system starting (it shouldn't be) then you can set the last two numbers of the relevant entry in your fstab file as '0'. For example:

# /etc/fstab: static file system information.
[clip]
# /dev/sda9 -- converted during upgrade to edgy
UUID=44F6-7DE3 /media/storage vfat defaults,utf8,umask=000,gid=46 0 0

"UUID=44F6-7DE3" is the strange way my system refers to my vfat partition mounted at /media/storage. If I remeber rightly, the final two numbers, both zero in this example, tell the system to continue booting if it can't find this partition (the first zero) and to not check it's integrity (the second zero)

Remember that you will get no warnings if your vfat partition becomes corrupt if you use this solution. There are also packages for configuring how often a specific partition is checked. You can find more about those in the ubuntu-users mailing list.

[Onno Benschop (onno-itmaze)]

FYI, Stefan Potyra and I are working through bugs related to dosfsck. There is currently discussion in progress to determine the issue you raise Duncan. You can find more information at bug #48806.

[Onno Benschop (onno-itmaze)]

I'm assigning this bug to me, seeing that Stefan and I are working through these kinds of issues.

[Daniel Werner (demitsu)]

Duncan wrote:
> But since fat doesn't support it I suggest we don't try and make it make complicated than necessary

With hindsight, I'll have to reluctantly agree. Kludging even more proprietary enhancements onto FAT has the potential to create more problems than it solves, especially because Windows would have ignored the "magic fsck record" anyway.

> Write a simple hidden text file with configuration info onto the fat partition. I might be called .mount-conf .

I am concerned it would be unneccessarily difficult to access .mount-conf on an up to this point *unmounted* file system, especially if said file system might be in an inconsistent state, in which case we'd have to run fsck first, which has to access .mount-conf -- a chicken-and-egg problem.

> The boot sequence could probably be configured to run dosfsck at certain intervals of uptime.

After the boot process has completed, the FAT partitions are probably mounted. I don't think dosfsck supports repairing live FAT volumes.

> If I remeber rightly, the final two numbers, both zero in this example, tell the system to continue booting if it can't find this partition (the first zero)

According to fstab(5), it's a signal for dump whether this partition needs or needs not to be dumped.

> and to not check it's integrity (the second zero)

This workaround works as long as users always cleanly unmount their filesystems. Which, in case of e.g. a kernel panic, is not given. Not checking at all and risking creeping integrity damage seems worse to me than checking too often. So, should we advise users to take this risk?

[Onno Benschop (onno-itmaze)]

There is a "dirty" flag on a FAT partition, you can read about it here:

http://www.geocities.com/thestarman3/DOS/DirtyShutdownFlag.html

Stefan and I have been discussing the implementation of this but decided to ask the ubuntu-devel list as well as upstream about their opinion on the matter.

Upstream has not yet responded and the ubuntu-devel message is still awaiting moderation.

As for the implementation of any other things. At this stage I'd be completely reluctant to create any configuration files on the FAT partition.

[Jason Spiro (jasonspiro)]

Did upstream reply yet? If upstream is unresponsive, perhaps you or I could modify the relevant tools to deal with the dirty flag. Hopefully this important issue can be resolved in time for Feisty.

As for checking every 30 mounts or so: Windows doesn't do it, and Windows is maintained by competent coders, so I think we'll be fine if we don't do it either.

* Do we really need to do it?

* And if we do it, how would we keep track of maximal mount count? (Will /etc usually be writable at mounting time? Could we keep track of the number of past mounts for each fat32 partition in a file in /etc?)

[Onno Benschop (onno-itmaze)]

At this stage I have had no reply from:

    * Upstream
    * ubuntu-devel
    * ubuntu-devel-discuss

I have attempted to initiate direct conversation with some senior developers with no response - they've been pretty busy with sprints.

I should point out that the running of dosfsck on each boot has a number of issues associated with it:

    * There are actual bugs in dosfsck which we're trying to fix
    * I have not located any docs related to Linux FAT dirty flags
    * I am writing specs to attempt to get further interest
    * I personally think the issue is the bugs, not the check

I have left this bug open because it is an issue to fix, but I do not believe that it has the highest priority - the dirty flag is an implementation matter, not a bug in itself IMHO.

I also do not think that it is appropriate that Ubuntu stores any flags or mount counts on the FAT volume. There should be IMHO an overall configuration on the booting Linux system.

If you're interested in assisting, check out the other bugs I have assigned to me and let me know what you think.

[Onno Benschop (onno-itmaze)]

A specification has been drafted to address the issue of checking a file system with a dirty flag.

The specification tracker is here:
* https://blueprints.launchpad.net/ubuntu/+spec/dos-dirty-flag

The specification is here:
* https://wiki.ubuntu.com/SpecificationDosDirtyFlag

[Jason Spiro (jasonspiro)]

Cool, thanks for writing a spec!

But maybe we should edit the spec to have it recommend the changes we
would need made to the kernel for optimal dirty flag support?

[Lorenzo (deodato-art)]

Will this bug ever be fixed? I have been using Ubuntu since the very first release and this issue has always been around. Right now if I don't disable boot check on my FAT partition (by editing fstab) I get a 60-second slowdown in the boot process every time I power up the machine.

I see three problems here:
1) The slowdown is such that may negatively affect the experience of users new to Ubuntu and linux.
2) There is no clear message explaining what is going on, so the newbie user doesn't have a clue about why the pc is taking so long to boot.
3) Altering fstab is not a real fix, because it means that the filesystem is never checked. Besides, editing fstab it is not trivial for an inexperienced linux user.

I think FAT is still quite widespread, especially among people with a dual-boot setup. Is there any plan to fix this issue?

[Jason Spiro (jasonspiro)]

Dear Philip Susi, why did you mark this bug report as a duplicate of bug #48806?

[Jason Spiro (jasonspiro)]

== Dosfstools and reading the dirty flag ==

A few changes[1] to dosfstools, made just before 3.0.14 was released, mean that dosfstools can now read a FAT partition's boot sector's dirty flag.

== Do all three major OSes set the dirty flag? ==

Windows versions since at least[2] Windows 2000 set the dirty flag in a FAT partition's boot sector when needed. Recent[2] released versions of the Linux kernel do too.

What do recent versions of Mac OS X do? How do you know?

== My proposal ==

If we discover that all three major OSes can set the dirty flag when needed, then:

A) I think someone should email the dosfstools maintainer and ask if he could please subscribe to this bug.
B) I suggest that dosfsck should not check a clean filesystem unless someone passes a "force" command-line parameter.

^ [1]. http://daniel-baumann.ch/gitweb/?p=software/dosfstools.git;a=shortlog
^ [2]. http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=b88a105802e9aeb6e234e8106659f5d1271081bb

[Jason Spiro (jasonspiro)]

> B) I suggest that dosfsck should not check a clean filesystem unless
> someone passes a "force" command-line parameter.

On second thought, this would be foolish. Dosfsck can never know for sure that a filesystem is truly clean; perhaps it was recently mounted by a smartphone which failed to set the dirty flag.

Instead:

C) A new "express" option should be added to dosfsck. When the "express" option is used, dosfsck should do the following. If a filesystem's dirty flag is set, dosfsck should check the filesystem. If the flag is unset, the filesystem is probably clean; dosfsck should do nothing.
D) When the OS runs dosfsck at startup, it should use the "express" option.