tools should use libvolume-id instead of libblkid

[Mikael Ståldal]

Edited by Chris Coulson {
vol_id is reporting an EXT3 partition as FAT16, which conflicts with the output of blkid. The partition in question is the root partition. Although the system is working okay, the root partition is not listed as being mounted by 'mount -l' or 'df -h'.

Some information:
1. lsb_release -rd
Description: Ubuntu 8.04
Release: 8.04

2. cat /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0

# /dev/sda1 -- converted during upgrade to edgy
UUID=07D6-050B / ext3 defaults,errors=remount-ro,user_xattr 0 1
# /dev/sda5 -- converted during upgrade to edgy
UUID=4766623a-8f3c-4ced-952c-f94333f9f311 none swap sw 0 0

3. df -h
Filesystem Size Used Avail Use% Mounted on
varrun 1,5G 104K 1,5G 1% /var/run
varlock 1,5G 0 1,5G 0% /var/lock
udev 1,5G 56K 1,5G 1% /dev
devshm 1,5G 0 1,5G 0% /dev/shm
lrm 1,5G 38M 1,5G 3% /lib/modules/2.6.24-17-generic/volatile

4. sudo mount -l
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
devshm on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
lrm on /lib/modules/2.6.24-17-generic/volatile type tmpfs (rw)
securityfs on /sys/kernel/security type securityfs (rw)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)

5. sudo blkid
/dev/sda5: TYPE="swap" UUID="4766623a-8f3c-4ced-952c-f94333f9f311"
/dev/sda2: TYPE="ufs"
/dev/sda1: UUID="48d6d975-d71b-45b5-a687-320148b8a1cd" SEC_TYPE="ext2" TYPE="ext3"

6. cat /proc/cmdline
root=UUID=07D6-050B ro quiet splash

7. ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 2008-05-29 11:35 07D6-050B -> ../../sda1
lrwxrwxrwx 1 root root 10 2008-05-29 11:35 4766623a-8f3c-4ced-952c-f94333f9f311 -> ../../sda5

8. sudo vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=vfat
ID_FS_VERSION=FAT16
ID_FS_UUID=07D6-050B
ID_FS_UUID_ENC=07D6-050B
ID_FS_LABEL=DellUtility
ID_FS_LABEL_ENC=DellUtility
ID_FS_LABEL_SAFE=DellUtility

9. sudo tune2fs -l /dev/sda1
tune2fs 1.40.8 (13-Mar-2008)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 48d6d975-d71b-45b5-a687-320148b8a1cd
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 27721728
Block count: 55420225
Reserved block count: 2771011
Free blocks: 50298999
Free inodes: 27241597
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Last mount time: Thu May 29 12:39:47 2008
Last write time: Thu May 29 12:39:47 2008
Mount count: 30
Maximum mount count: 30
Last checked: Wed Apr 30 09:02:35 2008
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 17432610
Journal backup: inode blocks

10. sudo fdisk -l
Disk /dev/sda: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot Start End Blocks Id System
/dev/sda1 1 27598 221680903+ 83 Linux
/dev/sda2 * 27599 30145 20458777+ bf Solaris
/dev/sda3 30146 30394 2000092+ 5 Extended
/dev/sda5 30146 30394 2000061 82 Linux swap / Solaris

} End edit by Chris Coulson

Original report:
When you mount a HDD partition via UUID, which the Ubuntu install program does by default, that file system does not show up in gnome-system-monitor, in df or in mount.

[Chris Coulson (chrisccoulson)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Unfortunately we can't fix it, because your description didn't include enough information. You may find it helpful to read "How to report bugs effectively" [WWW] http://www.chiark.greenend.org.uk/~sgtatham/bugs.html. We'd be grateful if you would then provide a more complete description of the problem.

We have instructions on debugging some types of problems at [WWW] http://wiki.ubuntu.com/DebuggingProcedures

At a minimum, we need:
1. Your current Ubuntu version (lsb_release -rd)
2. Your fstab
3. Output of 'df -h'
4. Output of 'sudo mount -l'
5. Output of 'sudo blkid'

A screenshot from gnome-system-monitor might help as well

Thanks!

[Mikael Ståldal (mikaelstaldal)]

1. lsb_release -rd
Description: Ubuntu 8.04
Release: 8.04

2. cat /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0

# /dev/sda1 -- converted during upgrade to edgy
UUID=07D6-050B / ext3 defaults,errors=remount-ro,user_xattr 0 1
# /dev/sda5 -- converted during upgrade to edgy
UUID=4766623a-8f3c-4ced-952c-f94333f9f311 none swap sw 0 0

3. df -h
Filesystem Size Used Avail Use% Mounted on
varrun 1,5G 104K 1,5G 1% /var/run
varlock 1,5G 0 1,5G 0% /var/lock
udev 1,5G 56K 1,5G 1% /dev
devshm 1,5G 0 1,5G 0% /dev/shm
lrm 1,5G 38M 1,5G 3% /lib/modules/2.6.24-17-generic/volatile

4. sudo mount -l
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
devshm on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
lrm on /lib/modules/2.6.24-17-generic/volatile type tmpfs (rw)
securityfs on /sys/kernel/security type securityfs (rw)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)

5.
/dev/sda5: TYPE="swap" UUID="4766623a-8f3c-4ced-952c-f94333f9f311"
/dev/sda2: TYPE="ufs"
/dev/sda1: UUID="48d6d975-d71b-45b5-a687-320148b8a1cd" SEC_TYPE="ext2" TYPE="ext3"

[Chris Coulson (chrisccoulson)]

Your fstab is messed up. Swap isn't mounting because the UUID specified doesn't exist, and the UUID for your root filesystem is invalid. I can only assume that your root filesystem is mounting due to the kernel option specified in your menu.lst.

What is the output of 'cat /proc/cmdline'?

Thanks

[Mikael Ståldal (mikaelstaldal)]

# cat /proc/cmdline
root=UUID=07D6-050B ro quiet splash

[Mikael Ståldal (mikaelstaldal)]

Then there must be a bug in the upgrade to Ubuntu 7.10, the messed up fstab was created by it.

[Chris Coulson (chrisccoulson)]

I'm still a bit confused actually. It seems that the UUID does exist, because your root filesystem is obviously being mounted by it, but blkid doesn't show it. What does 'sudo mount -a' say, and also 'ls -l /dev/disk/by-uuid/'?

[Mikael Ståldal (mikaelstaldal)]

"sudo mount -a" does not say anything.

# ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 2008-05-29 11:35 07D6-050B -> ../../sda1
lrwxrwxrwx 1 root root 10 2008-05-29 11:35 4766623a-8f3c-4ced-952c-f94333f9f311 -> ../../sda5

[Chris Coulson (chrisccoulson)]

Right, that's useful. blkid is giving one UUID, but udev thinks it is something else. I'll take another look at this later once I've worked out what other information will be useful from you

[Chris Coulson (chrisccoulson)]

Actually, for completeness, what is the output of 'sudo vol_id /dev/sda1'?

Thanks

[Mikael Ståldal (mikaelstaldal)]

# sudo vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=vfat
ID_FS_VERSION=FAT16
ID_FS_UUID=07D6-050B
ID_FS_UUID_ENC=07D6-050B
ID_FS_LABEL=DellUtility
ID_FS_LABEL_ENC=DellUtility
ID_FS_LABEL_SAFE=DellUtility

# sudo tune2fs -l /dev/sda1
tune2fs 1.40.8 (13-Mar-2008)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 48d6d975-d71b-45b5-a687-320148b8a1cd
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 27721728
Block count: 55420225
Reserved block count: 2771011
Free blocks: 50298999
Free inodes: 27241597
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Last mount time: Thu May 29 12:39:47 2008
Last write time: Thu May 29 12:39:47 2008
Mount count: 30
Maximum mount count: 30
Last checked: Wed Apr 30 09:02:35 2008
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 17432610
Journal backup: inode blocks

[Chris Coulson (chrisccoulson)]

Thats even stranger. vol_id thinks /dev/sda1 is a FAT16 partition. What is the output of 'sudo fdisk -l'?

[Mikael Ståldal (mikaelstaldal)]

# sudo fdisk -l
Disk /dev/sda: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot Start End Blocks Id System
/dev/sda1 1 27598 221680903+ 83 Linux
/dev/sda2 * 27599 30145 20458777+ bf Solaris
/dev/sda3 30146 30394 2000092+ 5 Extended
/dev/sda5 30146 30394 2000061 82 Linux swap / Solaris

[Chris Coulson (chrisccoulson)]

Assigning to udev, confirming, and I'm going to edit the title to something more appropriate.

Thanks

[Mikael Ståldal (mikaelstaldal)]

My computer is a Dell Optiplex GX620.

I think it had some kind of special "diagnostic" partition on the HDD when delivered. Perhaps this is what confuses vol_id, it says it has label "DellUtility".

I did originally installed Ubuntu 6.06, and I have done several dist-upgrades to reach 8.04.

[Chris Coulson (chrisccoulson)]

Yeah, I noticed the label of /dev/sda1 pointed it to being a Dell partition. I don't think it should make a difference though. It's a bit strange!

[Scott James Remnant (Canonical) (canonical-scott)]

vol_id's heuristics are considered "better" than blkids, however they're still heuristics and the wrong answer sometimes occurs if you use some bare-bones formatting tools which don't take care to wipe the metadata of previous filesystems off the disk

[Mikael Ståldal (mikaelstaldal)]

Wouldn't it be a good idea to use the same algorithm for both vol_id and blkid so that you don't get two different UUIDs for the same partition?

[Scott James Remnant (Canonical) (canonical-scott)]

We're attempting to phase out blkid completely, migrating tools to vol_id and libvolumeid

Which tool still uses it?

[Mikael Ståldal (mikaelstaldal)]

For me, tune2fs -l gives the same UUID as blkid, which is different from vol_id.

# sudo tune2fs -l /dev/sda1
tune2fs 1.40.8 (13-Mar-2008)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 48d6d975-d71b-45b5-a687-320148b8a1cd

# sudo blkid /dev/sda1
/dev/sda1: UUID="48d6d975-d71b-45b5-a687-320148b8a1cd" SEC_TYPE="ext2" TYPE="ext3"

# sudo vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=vfat
ID_FS_VERSION=FAT16
ID_FS_UUID=07D6-050B
ID_FS_UUID_ENC=07D6-050B
ID_FS_LABEL=DellUtility
ID_FS_LABEL_ENC=DellUtility
ID_FS_LABEL_SAFE=DellUtility

[Scott James Remnant (Canonical) (canonical-scott)]

See attached patch from SuSE

[Theodore Ts'o (tytso)]

Note that libvolume_id BREAKS ext4 support. For that reason the e2fsprogs patch HAS NOT BEEN ACCEPTED IN MAINLINE.

I am telling people who are using ubuntu to back out the changes in mount that switch it to use volume_id, as they are problematic for ext4, and recompile mount if they want to use ext4.

[Theodore Ts'o (tytso)]

PS, I'll note the original bug that started this chain was started because libvolume_id gave the wrong answer, erroneously reporting an ext3 filesystem as FAT. I've tried working with the vol_id folks before, and they are not interested in patches to make libvolume_id give more correct answers, because "in-band signally is always frought with potential errors", and so they aren't interested in reordering probes based on certain probes being more careful than others.

At that point, I gave up on the volume_id folks, since they aren't interested in fixing bugs. blkid I believe will always be more authoratative than libvolume_id --- and as I mentioned, there are specific probe functions which are required for ext4 support that are not, as far as I know, in libvolume_id.

[Scott James Remnant (Canonical) (canonical-scott)]

And that would leave us with the situation where depending which tool you pass the UUID to, you'll get a different block device.

We don't want two libraries for doing the exact same thing, in different ways. We want one library only.

Right now, the library that has the most patches to use it is libvolume-id; and it's the library used by HAL and udev which do the actual mounting these days. If you believe blkid is better, could you provide patches to udev and HAL to use that instead?

[Theodore Ts'o (tytso)]

When I have submitted patches in the past, to fix problems such as the one that kicked off this thread (an ext3 filesystem erroneously getting reported as a FAT filesystem), it was rejected by the libvolume_id maintainer.
I've been very busy trying to get ext4 production ready, and decided I didn't have time to waste trying to get them to listen to me. If you have an ext3 filesystem, and libvolume_id returns the FAT UUID, it is a bug with libvolume_id. Keeping them consistent may be slightly better, but returning the wrong type means that "mount /dev/sda1" won't work because mount will be given the wrong filesystem type by libvolume_id --- and the libvolume_id folks have rejected patches I've sent in the pass to correctly probe the device correctly.

So what the hell am I supposed to do?

[Scott James Remnant (Canonical) (canonical-scott)]

For every bug report someone files where their ext3 filesystem is reported as a FAT filesystem, somebody else files a bug where their FAT filesystem is reported as an ext3 filesystem.

Of course, that doesn't mean common sense could be applied. The priority should always be towards our favoured filesystems.

Don't suppose you still have those patches? I'll happily apply them ;)

[Julien Plissonneau Duquene (julien-plissonneau-duquene)]

Changing status to confirmed; apparently everyone agrees that something should be done.

[Scott James Remnant (Canonical) (canonical-scott)]

Already way ahead of you, this work has been done upstream but in the opposite direction - we've merged vol_id and blkid together into a libblkid library shipped by util-linux-ng. Packages exist in my PPA for testing, and we'll begin using this in karmic

[Mikael Ståldal (mikaelstaldal)]

blkid has the bad practice of updating a file in /etc (blkid.tab). This is annoying and against the spirit of <a href="http://www.pathname.com/fhs/pub/fhs-2.3.html#ETCHOSTSPECIFICSYSTEMCONFIGURATION">FHS</a>.

Can you fix that while you're at it?

[Scott James Remnant (Canonical) (canonical-scott)]

On Wed, 2009-04-15 at 19:16 +0000, Mikael Ståldal wrote:

> blkid has the bad practice of updating a file in /etc (blkid.tab). This
> is annoying and against the spirit of <a
> href="http://www.pathname.com/fhs/pub/fhs-2.3.html#ETCHOSTSPECIFICSYSTEMCONFIGURATION">FHS</a>.
>
If you test the packages in my PPA, you will see that this is fixed.

Scott
--
Scott James Remnant
<email address hidden>

[Scott James Remnant (Canonical) (canonical-scott)]

Fixed in Karmic where tools use libblkid from util-linux