Change of capitalization in filenames on FAT32

[Nicklas W Bjurman]

Binary package hint: nautilus, mv

It is not possible to change the capitialization of filenames on a FAT32 volume. This is a bug because, although FAT32 is not case-sensitive, it still have the ability to preserve the case of filenames.

Steps to reproduce:
1. Create file "test" on FAT32 volume
2. Try to rename new file to "TEST" using nautilus or mv

Actual results:
Displays an error 'The name "TEST" is already used in this folder. Please use a different name'

Expected results:
The filename should change to new requested filename

Does this happen every time?
Yes

[Colin Watson (cjwatson)]

(IMO this is a bug because, while FAT32 isn't case-sensitive, it is case-preserving.)

[Chris Coulson (chrisccoulson)]

Thanks, this looks like a glib bug though. Nautilus tries to change the filename with g_file_set_display_name_async () in both successful and unsuccessful attempts at changing the filename (I ran it through GDB to see), so it seems that the problem is in glib.

[Chris Coulson (chrisccoulson)]

Thank you for your bug report. This bug has been reported to the developers of the software. You can track it and make comments here: http://bugzilla.gnome.org/show_bug.cgi?id=567290

[Nicklas W Bjurman (lordmetroid)]

No, it is not glib that does this.
The terminal also shows the same behavior:

lordmetroid@lordmetroid-laptop:/media/disk$ mv test Test
mv: `test' and `Test' are the same file

It has to be in the implementation of the filesystem...

[Chris Coulson (chrisccoulson)]

When you ran "mv" there, it exited before even calling the rename() system call (I ran it through strace), so that could just be a bug in coreutils.

However, I just compiled a little test program (attached) and ran it through strace with some interesting results:

20:29:01 execve("./rename_file", ["./rename_file", "/media/disk/test", "/media/disk/Test"], [/* 38 vars */]) = 0 <0.012420>
20:29:01 brk(0) = 0x1880000 <0.000031>
20:29:01 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fad7a106000 <0.000038>
20:29:01 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.002623>
20:29:01 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fad7a104000 <0.000029>
20:29:01 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) <0.000024>
20:29:01 open("/etc/ld.so.cache", O_RDONLY) = 3 <0.000038>
20:29:01 fstat(3, {st_mode=S_IFREG|0644, st_size=112556, ...}) = 0 <0.000019>
20:29:01 mmap(NULL, 112556, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fad7a0e8000 <0.000025>
20:29:01 close(3) = 0 <0.000020>
20:29:01 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000023>
20:29:01 open("/lib/libc.so.6", O_RDONLY) = 3 <0.000032>
20:29:01 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\345"..., 832) = 832 <0.000022>
20:29:01 fstat(3, {st_mode=S_IFREG|0755, st_size=1502520, ...}) = 0 <0.000018>
20:29:01 mmap(NULL, 3609304, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fad79b77000 <0.000024>
20:29:01 mprotect(0x7fad79ce0000, 2093056, PROT_NONE) = 0 <0.000040>
20:29:01 mmap(0x7fad79edf000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x168000) = 0x7fad79edf000 <0.000037>
20:29:01 mmap(0x7fad79ee4000, 17112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fad79ee4000 <0.000027>
20:29:01 close(3) = 0 <0.000017>
20:29:01 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fad7a0e7000 <0.000023>
20:29:01 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fad7a0e6000 <0.000020>
20:29:01 arch_prctl(ARCH_SET_FS, 0x7fad7a0e66e0) = 0 <0.000018>
20:29:01 mprotect(0x7fad79edf000, 16384, PROT_READ) = 0 <0.000024>
20:29:01 mprotect(0x600000, 4096, PROT_READ) = 0 <0.000025>
20:29:01 mprotect(0x7fad7a107000, 4096, PROT_READ) = 0 <0.000023>
20:29:01 munmap(0x7fad7a0e8000, 112556) = 0 <0.000035>
20:29:01 rename("/media/disk/test", "/media/disk/Test") = 0 <0.000059>
20:29:01 exit_group(0) = ?

You'll notice that the rename() exits with success, but the filename never changed. So, there is a bug in the kernel.

[Chris Coulson (chrisccoulson)]

Thanks Nicklas

[Nicklas W Bjurman (lordmetroid)]

There was some confusion when I talked to other people if renaming files in this manner actually was possible to do on Windows. Just want to mention that it is possible to rename a file from test to Test on Windows XP.

[Nicklas W Bjurman (lordmetroid)]

I posed a question where to begin digging on the Linux kernel mailing list, the filesystem maintainer, OGAWA Hirofumi replied as below:

This is known issue for case-insensitive fs. (BTW, CI is case-insensitive).

To allow rename("filename", "FILENAME"), we have to fix two issues. One
is vfs_rename(), it doesn't call ->rename handler if inodes was same.

Another one is fs handlers, it has to handle dentry->d_name change while
someone is using it, and/or it has to care about dentry alias. Because,
the inode of both of names is same, and dentry should/may be same.

BTW, those is kernel side issues. There is some userland issues too
(e.g. IIRC, "mv" command check stat->st_ino before call rename(2)).

I also believe http://lxr.linux.no/linux+v2.6.28.2/fs/fat/namei_vfat.c#L53 is a key element to this bug.

[Loïc Minier (lool)]

Nicklas, great; do you know whether there are any upstream bugs in coreutils or linux tracking these?

[Chris Coulson (chrisccoulson)]

Thanks for looking in to this Nicklas. One thing to point out though - when you discover that the bug affects another source package in Ubuntu, you should click on the "Also affects distribution" link above, select Ubuntu and then specify the name of the source package. This way, the bug is assigned to the Ubuntu source package and can be tracked in Ubuntu. The extra tasks you added (by clicking on "Also affects project") are upstream tasks which we can use for tracking upstream bugs, but those tasks aren't tracked in Ubuntu.

[C de-Avillez (hggdh2)]

This has been discussed on the coreutils mailing list some time ago:

- http://lists.gnu.org/archive/html/bug-coreutils/2007-08/msg00042.html - no real conclusion, except that there was an agreement that there were doubts on the behaviour;

- http://lists.gnu.org/archive/html/bug-coreutils/2007-06/msg00207.html, where the suggestion is to do a two-way mv: mv a B; mv B A;

Also, given that we currently have a block on the kernel (on comment 8 above), I really do not see much sense on adding an upstream task for coreutis; in fact, I see no reason for upstream tasks that are not linked to an upstream bug (or whatever). Back to the problem, even if 'mv' supported it right now, the kernel issue would still block it.

Personally, I would rather have it work, though.

[Jim Lieb (lieb)]

It is correct that the kernel is that gating issue for making this change. There are technical issues that make the change difficult. The rename syscall is defined to be atomic to fix race and consistency problems with the original link+unlink behavior. Making an attempt to fix it in userspace, such as glibc or the application(s) breaks this requirement because the implementation would be equivalent to the original link+unlink. In addition, vfat requires an 8.3 alias name to also be created for compatiblility with the even older FAT filesystem. This makes it even more ugly. The 8.3 alias can be hidden in the vfat layer but it can have only one long name associated with it. The layers above don't know anything about this alias or the case insensitivity because, for the multiple other filesystems, it doesn't have to. All of these issues are solvable problems but a correct solution would have to invade all of the other filesystems and the upper (vfs) layer with case sensitive/insensitive state to work. This gets very ugly very fast.

The long and short of it is that the way UNIX/Linux handles case-insensitive filesystems is a compromise and every compromise involves giving up something. Making an atomic rename work correctly and cleanly in a pre-emptible, multi-cpu kernel where case sensitivity is the deeply embedded rule requires some things to go over the side, the issue in this bug being one of them. All in all, the compromises UNIX/Linux has had to make to accomodate these filesystems (VMS, MS, etc) have been better than the ones these other systems have tried to make to accomodate UNIX/Linux.

The chance of getting this change in the kernel is extremely slim on a number of grounds. The filesystem maintainers argued this down on technical merit 10+ years ago and have no interest in reopening the issue again. Without their support, and by extension, Linus's support, such a change would go nowhere. Some things we just have to live with.

[Chris Coulson (chrisccoulson)]

Based on Jim's comment, the user-space tasks for this bug should be closed too