findfs/blkid detects ext3 partition as ntfs (without UUID)

[Daniel Hahler]

I have dist-upgraded from Dapper => Edgy => Feisty.

I have a boot partition /dev/sda1:
# fdisk -l /dev/sda | grep sda1
/dev/sda1 * 1 7 56196 83 Linux

This is displayed in fstab as:
# /dev/hda1 -- converted during upgrade to edgy
UUID=70b7d252-5a65-45cd-a949-30569a2cce39 /boot ext3 defaults 0 2

# blkid /dev/sda1
/dev/sda1: TYPE="ntfs"

# vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=ext3
ID_FS_VERSION=1.0
ID_FS_UUID=70b7d252-5a65-45cd-a949-30569a2cce39
ID_FS_LABEL=
ID_FS_LABEL_SAFE=

This makes update-grub fail:
# LANG=C update-grub
Searching for GRUB installation directory ... found: /boot/grub
findfs: Unable to resolve 'UUID=70b7d252-5a65-45cd-a949-30569a2cce39'
Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Found kernel: /boot/vmlinuz-2.6.20-15-server
Updating /boot/grub/menu.lst ... done

Because update-grub fails to resolve the UUID, it puts "/boot/..." into the menu.lst and then grub itself fails with "File not found" (because it should be "/..." instead of "/boot/...").

I'm not sure where the bug here really is.. maybe there's some confusion about "hda1" and "sda1"?!

[Daniel Hahler (blueyed)]

# ls -l /dev/disk/by-uuid
...
lrwxrwxrwx 1 root root 10 2007-04-26 04:23 70b7d252-5a65-45cd-a949-30569a2cce39 -> ../../sda1
lrwxrwxrwx 1 root root 10 2007-04-26 04:23 88c84c55-6140-4fe4-ab56-66e6d9e1532a -> ../../sda3
lrwxrwxrwx 1 root root 10 2007-04-26 04:23 f52f2bea-f734-479d-a53a-4a86a9972b19 -> ../../sda2

[Daniel Hahler (blueyed)]

dumpe2fs displays the same/correct UUID (in contrast to what happens in bug 93921)

# dumpe2fs /dev/sda1 | grep UUID
dumpe2fs 1.40-WIP (14-Nov-2006)
Filesystem UUID: 70b7d252-5a65-45cd-a949-30569a2cce39

[Daniel Hahler (blueyed)]

Not only update-grub fails, but also booting in general. This is what I get during bootup of the server(!):

...
 * Will now check all file systems
fsck 1.40-WIP (14-Nov-2006)
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /boot] fsck.ext3 -a -C0 UUID=70b7d252-5a65-45cd-a949-30569a2cce39
fsck.ext3: Unable to resolve 'UUID=70b7d252-5a65-45cd-a949-30569a2cce39'
fsck died with exit status 8
 * File system check failed.
A log is being saved in /var/log/fsck/checkfs if that location is writable.
Please repair the file system manually.
 * A maintenance shell will now be started.
CONTROL-D will terminate this shell and resume system boot.
Give root password for maintenance
(or type Control-D to continue):
...

[Daniel Hahler (blueyed)]

The error appears to be that blkid does not detect the partition correctly and findfs checks against blkid.

blkid fails to report the UUID, which is available e.g. through vol_id and additionally says that it's a ntfs partition!

# fdisk -l /dev/sda
Disk /dev/sda: 61.4 GB, 61492838400 bytes
255 heads, 63 sectors/track, 7476 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot Start End Blocks Id System
/dev/sda1 * 1 7 56196 83 Linux
/dev/sda2 8 71 514080 83 Linux
/dev/sda3 72 7476 59480662+ 83 Linux

# blkid
/dev/sda1: TYPE="ntfs"
/dev/sda2: UUID="f52f2bea-f734-479d-a53a-4a86a9972b19" TYPE="swap"
/dev/sda3: UUID="88c84c55-6140-4fe4-ab56-66e6d9e1532a" SEC_TYPE="ext2" TYPE="ext3"
/dev/mapper/sda3: UUID="88c84c55-6140-4fe4-ab56-66e6d9e1532a" SEC_TYPE="ext2" TYPE="ext3"
/dev/mapper/sda2: TYPE="swap" UUID="f52f2bea-f734-479d-a53a-4a86a9972b19"
/dev/mapper/sda1: TYPE="ntfs"

[Theodore Ts'o (tytso)]

How was this filesystem created? It's apparently idnetifying it as an NTFS filesystem because 3072 bytes from the beginning of the filesystem, the signature string "NTFS " appeared, and this confused the blkid library. However, mke2fs should have cleared that portion of the disk to avoid false posiives; and it's done so for quite a long time. I should probably make the NTFS check more paranoid, but it shouldn't be happening....

[Theodore Ts'o (tytso)]

Actually, that's wrong; blkid is looking at offset 3 from the beginning of the filesystem ,and seeing "NTFS ". That looks like a bug. Am investigating...

[Daniel Hahler (blueyed)]

The partition is quite old and was pre-installed on my old Debian webserver, which I've dist-upgraded to Hoary IIRC.

I've commented out the ntfs line in the blkid_magic struct (./lib/blkid/probe.c) and now it seems to work correctly:
# blkid /dev/sda1
/dev/sda1: UUID="70b7d252-5a65-45cd-a949-30569a2cce39" SEC_TYPE="ext2" TYPE="ext3"

So, as you've said, apparently the ntfs check is wrong.

But I wonder though: are ntfs partitions not supposed to have an UUID?

[Theodore Ts'o (tytso)]

OK, I've looked at this some more, and I think I can guess what happened.

The NTFS superblock is located in the boot sector (the first 512-byte sector on the disk). Code to zap the bootsector has been in mke2fs for a long time (since at least 2000, and probably earlier). However, in 2002 we added code to avoid zapping the bootsector if it looked like a BSD disk label, since that cause problems. I'm guessing that you had a partifion which previously had contained NTFS, or which at any rate had the NTFS signature at offset 3 from the beginning of the boot sector, but which also looked like BSD disk label, so mke2fs avoided clearing out the boot sector. This allowed blkid to misidentify the filesystem.

The enclosed patches, which will be in e2fsprogs 1.40, tries harder to validate that the partition really is NTFS, and sets the NTFS UUID and Label information. It still can get fooled, since the MFT is located near the middle of the disk, so it won't necessarily get zapped by mke2fs.

I will also probably change mke2fs so that it will be more aggressive about zapping the boot sector if it contains an NTFS signature.

[Daniel Hahler (blueyed)]

I can confirm that the patch works. Thanks a lot!

Should I provide a debdiff for Gutsy/Feisty?

[Theodore Ts'o (tytso)]

Stupid question, but what's the schedule for Gutsy? If I get e2fsprogs 1.40 released in the next week or so, would it be too late for Gutsy to move up to the e2fsprogs 1.40 release?

[Daniel Hahler (blueyed)]

This would be ok, as far as I know. See https://wiki.ubuntu.com/GutsyReleaseSchedule

As for Feisty, I will provide a debdiff.
Given the other related bugs, which you have commented on as "probably dupe of this one" and the importance of this, it makes sense to have the fix in -updates.

[Theodore Ts'o (tytso)]

Note: e2fsprogs 1.40-rc1 was released yesterday at:

http://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/e2fsprogs-1.40-rc1.tar.gz

E2fsprogs 1.40 final is scheduled to be released at the end of the week; if folks could test it to make sure there aren't any seriously embarassing bugs, I would greatly appreciate it. I am currently running e2fsprogs 1.40-rc1 on my Ubuntu Feisty laptop....

If anyone can connect me with whomever makes e2fsprogs decision making for Ubuntu Gutsy about getting this into the Gutsy (since I believe I just missed the last Debian import deadline), I would greatly appreciate it.

[Theodore Ts'o (tytso)]

FYI, e2fsprogs 1.40 has been released, both in upstream, and I just uploaded e2fsprogs 1.40-1 to Debian/sid (currently in upload queue as of this writing).

Can someone let me know what's necessary to get the Gutsy release engineering team to consider pulling this into gutsy? Thanks!!

[Gerald Murray (gmurray)]

e2fsprogs-1.40 released upstream. Allege possible corruption to important file backup merits
assigning new version to backporting for prior ubuntu releases. Thanks for consideration.

[Theodore Ts'o (tytso)]

Note that if Gutsy is going to take e2fsprogs, the version that should be taken at this point is e2fsprogs 1.40.1, which has a number of important bugfixes over e2fsprogs 1.40.

[Daniel Hahler (blueyed)]

Thanks Theodore. I will look into providing a debdiff for Gutsy.

[Daniel Hahler (blueyed)]

I have looked at the merge from the Debian and the remaining Ubuntu change can be dropped, so I filed a sync request (bug 125331).

[Daniel Hahler (blueyed)]

Fixed in 1.40-1, which has been synced (as 1.40.2-1) to Gutsy.

[Daniel Hahler (blueyed)]

I'm not sure, if this bug justifies an update for Feisty.
The debdiff for this is attached already however.

[Daniel Hahler (blueyed)]

Please apply the already provided debdiff for feisty-proposed.

The impact seems quite high, seeing people reporting problems, which may be related to this bug and having it in feisty-proposed would allow those to test and confirm it.

The bug has been fixed upstream and for Gutsy already. The raw patch is also attached to this bug.

The regression potential is quite high, given that it can lead to a boot failure, if there's a new problem with a currently working system because of this patch.

Theodore, can you confirm that there has not been problem reported against e2fsprogs because of this patch and that it's a good candidate for a StableReleaseUpdate?

[JasonVenkiteswaran (jjvenky)]

FYI:

I've upgraded from 6.06 to 8.04 (that's dapper to hardy) and had this problem.

I had the same fsck error as reported above. At the maintenance shell mount -a mounted all the parititions and them CTRL+D would continue to boot.

In short, my /dev/sda1 is /boot. The fstab was updated with UUIDs. But blkid reported a short UUID and the wrong filesystem for this partition:
blkid /dev/sda1
/dev/sda1: UUID="7EFA-09BD" SEC_TYPE="msdos" TYPE="vfat"

This doesn't make sense because /dev/sda1 is ext3:
fdisk /dev/sda
   Device Boot Start End Blocks Id System
/dev/sda1 * 1 31 248976 83 Linux

findfs reported correctly with a short UUID, but would not with the longer UUID:
findfs UUID="7EFA-09BD"
/dev/sda1
findfs UUID="a1a3a3ef-cea0-4230-8098-070febf47f80"
findfs: Unable to resolve 'UUID=a1a3a3ef-cea0-4230-8098-070febf47f80'

However, fstab had this longer UUID=a1a3a3ef-cea0-4230-8098-070febf47f80 and worked fine with mount -a.

Following Theodore Tso's advice from bug #66032, over here:
https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/66032

I used dd on that partition and it all works swimmingly now that fsck uses that longer UUID and blkid reports correctly:
blkid /dev/sda1
/dev/sda1: LABEL="/boot" UUID="a1a3a3ef-cea0-4230-8098-070febf47f80" SEC_TYPE="ext2" TYPE="ext3"

[LumpyCustard (orangelumpycustard)]

Please close for Feisty as Won't Fix? This goes for all the other Feisty bugs.

[Hew (hew)]

Ubuntu Feisty Fawn is no longer supported, so a SRU will not be issued for this release. Marking Feisty as Won't Fix.