dosfsck fixes files with "illegal file names"

[JohnL]

During installation, ubuntu checks all discs for errors and FINDS errors on VFAT drives that are actually clean.

During installation, Ubuntu correctly asks if the errors should be fixed and, of course, you say no.

Ubuntu then installs and reboots. On reboot, Ubuntu scans all drives using dosfsck set to automatically "fix" errors. The "errors" were all "illegal file name" and were all files/directories with (legal on VFAT) spaces in the name such as "C:\Program Files",
"My Documents".

Ubuntu only declared SOME files with spaces illegal, not All.

THIS IS A CRITICAL BUG AS IT CAUSES ubuntu TO ERRONEOUSLY MODIFY ALTERNATIVE OPERATING SYSTEM FILES.

In the short term:
If you MUST scan other operating system discs before mounting them, you should give users the option to "NOT FIX and Don't Mount" / "Not Fix and Mount anyway" / "Fix and risk the consequences"
Or perhaps just don't mount any disk ubuntu THINKS has errors.
In the long term you should fix dosfsck so that it doesn't pick up false errors on VFAT drives.

[Cody A.W. Somerville (cody-somerville)]

This appears to be fixed now. Please reopen if this is not the case.

[JohnL (cbarter)]

When was this fixed and how do I get the patch - I found the error Ubuntu 6.0.6 which I believe is your current release.

[Cody A.W. Somerville (cody-somerville)]

It appears that the bug was fixed in 5.10 and then reintroduced again in
6.06.

On 9/24/06, Geo Smith <email address hidden> wrote:
>
> *No, it is NOT fixed!!!* I have tried the latest version of Kubuntu
> 6.06.1, and it still checks all FAT/FAT32 partitions for errors, finds *
> "imaginary*" errors, and "fixes" them by truncating correct Windows files,
> thereby *destroying *them!!!
> The only way to fix the problem is:
> *1. Boot Knoppix and manually edit Ubuntu/Kubuntu /etc/fstab
> For each FAT entry in fstab, delete 1 in column 6
> 2. Save the modified fstab*
>
> *This is very easy to do using a sed script. *
>
> Then Kubuntu/Ubuntu no longer checks FAT32 partions and does* not* destroy
> them!
>
> You should post a patch for Ubuntu/Kubuntu 6.06 and 6.06.1 users as soon
> as possingle.
>
> Note: Ubuntu live CD does not auto-mount Ubuntu partitions that are
> installed on the hard drive. *This is a serions oversight*! *That's why
> Knoppix can be used to fix fstab, and Ubuntu live cd cannot.
> *
>
> ------------------------------
> From: *"Cody A.W. Somerville" <email address hidden>*
> Reply-To: *Bug 55121 <email address hidden>*
> To: *<email address hidden>*
> Subject: *[Bug 55121] Re: Ubuntu alters "other" file systems on startup
> withFATAL results*
> Date: *Fri, 01 Sep 2006 07:19:17 -0000*
> >*** This bug is a duplicate of bug 48806 ***
> >
> >This appears to be fixed now. Please reopen if this is not the case.
> >
> >** Changed in: Ubuntu
> > Status: Unconfirmed => Fix Released
> >
> >--
> >Ubuntu alters "other" file systems on startup with FATAL results
> >https://launchpad.net/bugs/55121
>
>
> ------------------------------
> Get today's hot entertainment gossip
> <http://g.msn.com/8HMAENUS/2755??PS=47575>

--

Firefox (www.getfirefox.com) -- A browser you can trust

[JohnL (cbarter)]

Editing fstab works fine. I agree with geo smith though - a patch should be posted ASAP.

[[e-mail] (geo99)]

The fix for bug 48806
   |
The attached one-line sed script can be used either to fix fstab generated
by Ubuntu/Kubuntu installer or it can be distributed as a patch for the
affected Ubuntu/Kubuntu versions that turn on the fsck check for fat/msdos
partitions. For example, for versions 6.06 and 6.10.
Note: I have not tested version 6.11.

# PURPOSE: Turn off the fsck test for vfat and msdos partitions in fstab
# INVOCATION: sed -i.bak -f fix-fstab.sed.txt /etc/fstab
# Note: You must run this script as an administrator!
# SCRIPT: fix-fstab.sed.txt
/^\/dev\/hda.*[ \t]\(vfat\|msdos\)[ \t]/s/^\(\([^ \t][^ \t]*[ \t][
\t]*\)\{5\}\)1/\10/

# ALTERNATIVE SCRIPT: A simpler but less safe version of the above script
#/^\/dev\/hda.*[ \t]\(vfat\|msdos\)[ \t]/s/1[ \t]*$/0/

I have tried to post this fix on Bugzilla, but it would not accept my login
credentials. Please include a copy of this message on Bugzilla so that the
developers could use it to create patch or to prevent fsk check in future
versions of Ubuntu/Kubuntu.

>From: JohnL <email address hidden>
>Reply-To: Bug 55121 <email address hidden>
>To: <email address hidden>
>Subject: [Bug 55121] Re: Ubuntu alters "other" file systems on startup
>withFATAL results
>Date: Sun, 01 Oct 2006 00:24:41 -0000
>
>*** This bug is a duplicate of bug 48806 ***
>
>Editing fstab works fine. I agree with geo smith though - a patch should
>be posted ASAP.
>
>--
>Ubuntu alters "other" file systems on startup with FATAL results
>https://launchpad.net/bugs/55121

_________________________________________________________________
Fixing up the home? Live Search can help
http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmemailtaglinenov06&FORM=WLMTAG

[Onno Benschop (onno-itmaze)]

Hi, thank you for your bug report.

We've just been trying to reproduce this bug, but couldn't. Is there anything you can tell us about your computer that might give more information. For example:

* Was a specific file, or group of files, involved?
* What names did affected files have?
* Were they all in the same directory, what was the name of that directory?
* How big was the partition that contained the files?
* Which language was the file system?
* Was the file of a particular size?
* Was the file a Windows .lnk file?
* What other symptoms did you notice?

We realise that we're asking lots of questions, but the bug you reported has potentially been identified as being part of a problem tucked away in the internal workings of the operating system and if that is the case, it may affect many users.

Thanks,
Stefan and Onno

[JohnL (cbarter)]

I find it hard to believe you cannot reproduce this bug!!!!!

All you have to do is install Ubuntu 6.06 or 6.10. THAT'S IT - 100%
success in reproducing the bug.

The bug is that Ubuntu checks other file systems - FULL STOP.

The fact that dosfsck only destroys the "other" file system
intermittently is possibly difficult to detect, BUT WHO CARES.

As no useful purpose is gained by checking "other" file systems, the fix
is easy.

Modify the fstab file to only check appropriate Ubuntu file systems on
boot. This is easily accomplished by setting column 6 in the fstab file
to 0.

As the Ubuntu installer produces the fstab file, it is a SIMPLE matter
for the developers to change the routine that builds the fstab file to
get it right.

It is also a simple matter to distribute a patch (or fix to prevent this
happening in the first place) as <email address hidden>
<https://launchpad.net/people/geo99> has been pointing out for ages.

Look, this is a really serious bug and I can't understand why you are
still distributing Ubuntu while this bug persists.

I would have thought it would be prudent to at least
  - issue a warning that the distribution should not be installed in a
dual boot situation,
  - modify the installer so as not to install in a dual boot situation

or simply recall all affected install discs so far distributed.

Perhaps this is the downside of open source - no fear of bankruptcy
compelling a company to do the right thing.

NOTE: Not WASTING TIME checking other filesystems for no reason, also
reduces bootup time markedly - from about 4 minutes to less than 1
minute in my system which has 8 VFAT drives!

John Lovely

[StefanPotyra (sistpoty)]

Hi,

this is in fact not one bug, but three:
1) dosfsck is buggy
2) other file systems are checked during boot
3) file system checks depend on a buggy program

I'm pretty much interested in fixing 1). So any useful info how reproduce this are highly appreciated.

As I'm most of the time just a user of ubuntu, I'm not interested in 2) and 3), because for me it just works. However here are some comments to these, but these reflect only my humble opinion:

2) please define "other file systems":
I have a ext3 FS which I use to store various data on. It's neither vital nor in any way necessary for ubuntu to work. Would this also fall under this catagory?
Imho there is only a point in making a distinction between rw-mounted file systems and ro-mounted ones.

3) that's a serious bug indeed, fixing the installer would fix that. Of course fixing 1) would also fix that.

I don't think that the warning thingy might get us anywhere closer. For me personally dosfsck never ate any files, so you'd make me unhappy if I suddenly couldn't write to my fat32 partition or in case it does get corrupted if it wouldn't be fixed on boot.
However other bugs have shown that dosfsck is indeed buggy, and with the help of the submitters I could provide a fix for some of these.

Thanks,
    Stefan aka sistpoty.

[JohnL (cbarter)]

For me the errors occurred when checking VFAT drives on startup. The
drives were not reported to have any errors during previous Windows Me
session.
Win Me "Scandisk" did find (and fix) lots of errors after being "fixed"
by dosfsck. In the process I lost 256 Mb of disk space.

All errors seemed to be with files with long file names including a
space somewhere in the path. eg C:\My Programs\MSACCESS.EXE or
F:\Letter to fred.doc

(I can't remember where that information came from - whether it was from
dosfsck reporting the errors it found or from Scandisk fixing dosfsck's
errors - I think I identified it in my original bug report)

During the installation, while preparing to partition the disk, the
installer "checks discs for errors" and, if it finds "errors", asks
politely if you want them fixed. If the installer passes this stage
without detecting errors, the install seems to succeed. If you find
errors and proceed to install without letting dosfsck fix the "errors",
at the first system reboot, dosfsck will "fix" the "errors" without asking.

On another installation attempt, dosfsck reported errors, I aborted the
install and ran scandisk under Windows. Windows DID find one error on
each disk that dosfsck identified and no errors on disks that dosfsck
declared to be OK.

So it appears that dosfsck might be correctly identifying errors but
making things worse when trying to fix them. Hope this narrows the
search for the bug(s).

So, it would seem possible to duplicate the bug by deliberately creating
errors in a VFAT file system, probably in files with spaces in the path.
It should be possible to create VFAT disc errors by opening and editing
several files in wordprocessors / spreadsheets etc then resetting the
computer (without proper shutdown).

I haven't had any problems with ReiserFS, ext3 or ext2 filesystems.

StefanPotyra wrote:

> *** This bug is a duplicate of bug 48806 ***
>
> Hi,
>
> this is in fact not one bug, but three:
> 1) dosfsck is buggy
> 2) other file systems are checked during boot
> 3) file system checks depend on a buggy program
>
> I'm pretty much interested in fixing 1). So any useful info how
> reproduce this are highly appreciated.
>
> As I'm most of the time just a user of ubuntu, I'm not interested in 2)
> and 3), because for me it just works. However here are some comments to
> these, but these reflect only my humble opinion:
>
> 2) please define "other file systems":
> I have a ext3 FS which I use to store various data on. It's neither vital nor in any way necessary for ubuntu to work. Would this also fall under this catagory?
> Imho there is only a point in making a distinction between rw-mounted file systems and ro-mounted ones.
>
> 3) that's a serious bug indeed, fixing the installer would fix that. Of
> course fixing 1) would also fix that.
>
> I don't think that the warning thingy might get us anywhere closer. For me personally dosfsck never ate any files, so you'd make me unhappy if I suddenly couldn't write to my fat32 partition or in case it does get corrupted if it wouldn't be fixed on boot.
> However other bugs have shown that dosfsck is indeed buggy, and with the help of the su...

[Onno Benschop (onno-itmaze)]

Thanks for your additional information John.

I'm trying to gather more information regarding your report to be able to fix the problem you describe.

The challenge we have in attempting to fix the problem you are experiencing is that we need to be able to reproduce it. We are not able to do that with the information you've provided.

Can you please attach the output of:
 dosfsck -v -n <partition>

And also give some information about the files, dosfsck wants to fix (e.g. length, filename, whether these files are indeed broken)?

[Marco Rodrigues (gothicx)]

Have this bug tested at Feisty Fawn ?

[Onno Benschop (onno-itmaze)]

John, we are closing this bug report as it lacks the information, described in the previous comments, we need to investigate the problem further. However, please reopen it if you can give us the missing information and don't hesitate to submit bug reports in the future.

[Onno Benschop (onno-itmaze)]

No further response received from initial reporter, bug rejected. If further information becomes available, please don't hesitate to re-open this report, or submit a new one.

[WC Leung (lwchk2001)]

Sorry to jump in a year later, bug this bug persists and is critically serious.

What I did is to install Ubuntu 7.10, booting the system, and then dosfsck scans my fat32 partitions.
It claimed to find some invalid filenames (which are completely valid) and rename it to invalid filename (containing the '\' character.)

The effect is that the file will be impossible to be opened or deleted in Windows. The scandisk utility in Windows is also not able to fix the mess.

The only possible way to clean this is to rename the affected files one by one using Linux.