read-only bind mounts cause spurious EROFS

[Peter Cordes]

I installed the Intrepid kernel (linux-image-2.6.27-5-server on an otherwise-Hardy x86-64 server (I do have Intrepid's initramfs-tools, and a non-current-Intrepid module-init-tools).

/dev being read-only bind mounted on /dev/.static/dev somehow causes Linux to return EROFS for only some operations on the root filesystem. (My root filesystem is XFS. Yes, this works, you just can't setup grub on it from linux, so I boot grub from USB or PXE after the initial install of Ubuntu.)

I don't have this problem on my pre-Intrepid desktop at home running 2.6.27-4-generic, with a JFS root filesystem (and /dev read-only bind mounted). And I didn't notice it on my pre-Intrepid laptop with 2.6.27-5-generic (IIRC), with an ext3 rootfs. It's not on, so I can't check it's status right now. So either a server kernel bug, or an XFS bug, I'd guess.

symptoms:
# strace touch /etc/bar # /etc/bar doesn't exist beforehand
close(0) = 0
open("/etc/bar", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 0
syscall_280(0, 0, 0, 0, 0x2, 0, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2, 0x2) = -1 (errno 30)
utimes("/etc/bar", NULL) = -1 EROFS (Read-only file system)
close(0) = 0
touch: setting times of `/etc/bar': Read-only file system
exit_group(1) = ?

/etc/bar now exists.

# strace chmod 644 /etc/mtab
stat("/etc/mtab", {st_mode=S_IFREG|0600, st_size=794, ...}) = 0
fchmodat(AT_FDCWD, "/etc/mtab", 0644) = -1 EROFS (Read-only file system)

/proc/mounts showed that / was mounted rw, but /dev/.static/dev was mounted ro.
mount /dev /dev/.static/dev -o remount,rw makes metadata ops work again on the root filesystem.

/proc/mounts (after remount)
rootfs / rootfs rw 0 0
/dev/disk/by-uuid/dc2372d6-5ae9-4b28-8fe5-108c779e82fb / xfs rw,relatime,noquota 0 0
/dev/disk/by-uuid/dc2372d6-5ae9-4b28-8fe5-108c779e82fb /dev/.static/dev xfs rw,noquota 0 0

/etc/fstab
# /dev/sda1
UUID=dc2372d6-5ae9-4b28-8fe5-108c779e82fb / xfs relatime,inode64,noikeep 0 1

It's a 10GB filesystem, so I probably don't need inode64...

Let me know if you have any problem reproducing this, and I can get more details on my server.

[Peter Cordes (peter-cordes)]

I was able to reproduce this on another server. It happens on XFS with relatime,inode64,noikeep. It does not happen on 2.6.27-5 with just relatime. I haven't checked which of inode64 or noikeep is the problem.

[Peter Cordes (peter-cordes)]

looks like this is fixed in Linux 2.6.27.3:
commit c068663ae65e507814545b59a8e2090f48a85613
Author: Christoph Hellwig <email address hidden>
Date: Sun Oct 12 14:30:44 2008 +0200

    xfs: fix remount rw with unrecognized options

[...]otherwise we may skip clearing
    the xfs-internal read-only flag. This will only show up on some
    operations like touch because most read-only checks are done by the VFS
    which thinks this filesystem is r/w.
[...]

[Leann Ogasawara (leannogasawara)]

Hi Peter,

I'm marking this as a duplicate of bug 273601. It's referenced the same patch and that bug has already been milestoned for intrepid-updates which basically means we'll be trying to get the patch pulled in and available in a post Intrepid kernel release. Thanks.

[Launchpad Janitor (janitor)]

Per a decision made by the Ubuntu Kernel Team, bugs will longer be assigned to the ubuntu-kernel-team in Launchpad as part of the bug triage process. The ubuntu-kernel-team is being unassigned from this bug report. Refer to https://wiki.ubuntu.com/KernelTeamBugPolicies for more information. Thanks.