'df /' reports Filesystem '-'

[Scott Moser]

somewhere between beta-2 and 20121001, we have a change in df output:

From:
 | $ cat /etc/cloud/build.info
 | build_name: server
 | serial: 20120925
 | $ df -h /
 | Filesystem Size Used Avail Use% Mounted on
 | /dev/vda1 9.9G 1.1G 8.5G 11% /
 | $ dpkg-query --show coreutils
 | coreutils 8.13-3.2ubuntu1

To:
 | $ cat /etc/cloud/build.info
 | build_name: server
 | serial: 20121002
 | $ df -h /
 | Filesystem Size Used Avail Use% Mounted on
 | - 9.9G 1.1G 8.5G 11% /
 | $ dpkg-query --show coreutils
 | coreutils 8.13-3.2ubuntu2

Related bugs:
 * mountall bug 643289: idmapd does not starts to work after system reboot

ProblemType: Bug
DistroRelease: Ubuntu 12.10
Package: coreutils 8.13-3.2ubuntu2
ProcVersionSignature: User Name 3.5.0-16.25-generic 3.5.4
Uname: Linux 3.5.0-16-generic i686
ApportVersion: 2.6.1-0ubuntu1
Architecture: i386
Date: Tue Oct 2 16:07:59 2012
Ec2AMI: ami-00000158
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: nova
Ec2InstanceType: m1.small
Ec2Kernel: unavailable
Ec2Ramdisk: unavailable
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: coreutils
UpgradeStatus: No upgrade log present (probably fresh install)

[Scott Moser (smoser)]

I'm guessing this is a result of the contents of /etc/mtab changing from:
$ cat /etc/mtab
/dev/vda1 / ext4 rw 0 0
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
none /sys/fs/fuse/connections fusectl rw 0 0
none /sys/kernel/debug debugfs rw 0 0
none /sys/kernel/security securityfs rw 0 0
udev /dev devtmpfs rw,mode=0755 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
none /run/shm tmpfs rw,nosuid,nodev 0 0
none /run/user tmpfs rw,noexec,nosuid,nodev,size=104857600,mode=0755 0 0
/dev/vdb /mnt ext3 rw,_netdev 0 0

to
$ cat /etc/mtab
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
udev /dev devtmpfs rw,mode=0755 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
/dev/vdb /mnt ext3 rw,_netdev 0 0

[Steve Langasek (vorlon)]

Not reproducible here. mountall includes explicit support for a second pass of 'mount -f' after the rootfs is mounted read-write, so mountall --verbose output is needed to diagnose this.

[Steve Langasek (vorlon)]

The mountall output here looks perfectly normal:

mount /sys/fs/fuse/connections [320] exited normally
[...]

mount /sys/kernel/debug [341] exited normally
[...]
mount /sys/kernel/security [344] exited normally
[...]
mount /run/lock [345] exited normally
[...]
mount /run/shm [346] exited normally
[...]
mount /run/user [347] exited normally
[...]
mounting event handled for /
remounting /
mount / [348] exited normally
[...]
mount /proc [350] exited normally
mount /sys [351] exited normally
mount /sys/fs/fuse/connections [352] exited normally
mount /dev [353] exited normally
mount /dev/pts [354] exited normally
mount /run [355] exited normally

Each of those repeated 'mount' commands following the remount of / is there to update /etc/mtab with mount -f. Are you sure nothing else on the system is editing the mtab?

[Scott Moser (smoser)]

I dont know what else would be hitting /etc/mtab. I can easily set up an instance for you to poke around at.
cloud-init does run 'mount -a' after modifying /etc/fstab.

both the modifying of /etc/fstab and the invoking of 'mount -a' happens during the cloud-config job, which is:
  start on (filesystem and started rsyslog)

I would have thought that suitably late to not screw stuff up with mountall.

[Steve Langasek (vorlon)]

tracked this down to a problem with how we handle the mount -f calls for filesystems mounted before the root fs has been mounted rw: since the mount is not considered "mounted" until the mounted event has returned, write_mtab() was missing writing entries for mounts that were still in flight.

[i3dmaster (i3dmaster-gmail)]

I can confirm this. Just update from 3.5.0.17 - #26-ubuntu to #27-ubuntu and I now don't see / on df anymore.

[Launchpad Janitor (janitor)]

This bug was fixed in the package mountall - 2.42

---------------
mountall (2.42) unstable; urgency=low

  [ Andy Whitcroft ]
  * Add support for mounting the efivars filesystem onto
    /sys/firmware/efi/efivars. LP: #1063061.

  [ Steve Langasek ]
  * mount events now happen so much in parallel that /etc/mtab may not get
    updated due to races between 'mounted' events. Explicitly track the
    list of mounts that are in need of fake-remounting and make sure mount -f
    gets called for them if they missed the boat. LP: #1060296

 -- Steve Langasek <email address hidden> Tue, 09 Oct 2012 12:32:56 -0700

[Scott Moser (smoser)]

This seems to me to be fixed correctly in quantal.
Thank you Steve.