mountall hangs in case of inconsistent /etc/fstab

[ooops]

Binary package hint: mountall

Due to some unrelated problem my /etc/fstab contained an entry to a UUID mount where /dev/disks/by-uuid was not available. Instead of giving an error mountall simply hangs. Probably due to starting cryptsetup stealing keyboard input (hint from another bugreport for mountall) it was not even possible to stop it with ctrl-c. Fortunately an alt-sysrequest-e killed mountall and dropped me to a shell where I could analyse the problem. This is definitely not a user-friendly error-behavior.

[Daniel Hahler (blueyed)]

Can you elaborate on your investigations?

I might have the same problem now, attaching screenshots.

[Daniel Hahler (blueyed)]

My /etc/fstab:

proc /proc proc defaults 0 0
UUID=73d091ef-8ee9-4ad2-9243-1ece878c1024 / ext3 defaults,errors=remount-ro,relatime 0 1
UUID=739ad8eb-d03e-4a76-a54d-4ac40107099d /boot ext2 defaults,errors=remount-ro,relatime 0 1
UUID=04ba6931-c4bb-499c-9ed0-b1f4f165b3c8 /home ext3 defaults,errors=remount-ro,relatime 0 2
UUID=6b6802a5-2647-4351-a28a-c22973d6067b /mnt/datalv ext3 defaults,relatime 0 2
/mnt/datalv /opt none bind
/mnt/datalv /mnt/data none bind
UUID=c768c319-5563-44ec-9364-c3072adf7bf9 /mnt/datanr ext4 defaults,relatime 0 2
UUID=9ce3c0e6-bb96-4451-b03b-0569b1dc2a77 none swap sw 0 0
UUID=05a79c7f-5236-449c-904d-dfb0a4738fd1 none swap sw 0 0
tmpfs /tmp tmpfs size=4g
/dev/hda1 /mnt/win/c ntfs umask=0222,nls=utf8,noauto 0 0
/dev/hda5 /mnt/win/d ntfs umask=0222,nls=utf8,noauto 0 0
/dev/hdc1 /mnt/extern ext3 defaults,noauto 0 0
none /mnt/gmail gmailfs noauto,<email address hidden>,fsname=XXX
sshfs#root@XXX:/home/www/XXX /home/XXX fuse defaults,user,noauto 0 0
none /proc/bus/usb usbfs devgid=143,devmode=664 0 0
/tmp/app/1/image /tmp/app/1 cramfs,iso9660 user,noauto,ro,loop,exec 0 0
/tmp/app/2/image /tmp/app/2 cramfs,iso9660 user,noauto,ro,loop,exec 0 0
/tmp/app/3/image /tmp/app/3 cramfs,iso9660 user,noauto,ro,loop,exec 0 0
/tmp/app/4/image /tmp/app/4 cramfs,iso9660 user,noauto,ro,loop,exec 0 0
/tmp/app/5/image /tmp/app/5 cramfs,iso9660 user,noauto,ro,loop,exec 0 0
/tmp/app/6/image /tmp/app/6 cramfs,iso9660 user,noauto,ro,loop,exec 0 0
/tmp/app/7/image /tmp/app/7 cramfs,iso9660 user,noauto,ro,loop,exec 0 0

[Daniel Hahler (blueyed)]

The devices not being mounted are cryptsetup/luks partitions, which are listed in /etc/crypttab, using a key-file.

I guess the problem is that mountall is waiting for them to become available, but they never do.

So, after all, I think the initial bug report is related, as mountall should fail/not hang with invalid UUIDs, but my problem goes one step further: the cryptsetup devices are not being setup properly anymore.
Note: the root device is mounted/opened correctly using cryptsetup via passphrase.

[Daniel Hahler (blueyed)]

Marking as Triaged, since I can confirm it (via other circumstances).
My comments above are not very relevant: I can imagine that given a invalid UUID in fstab, mountall would hang forever.

[Scott James Remnant (Canonical) (canonical-scott)]

I've uploaded a new mountall package to the ubuntu-boot PPA:

https://launchpad.net/~ubuntu-boot/+archive/ppa

I would appreciate it if you could install this and try it out. *BEFORE* you reboot though, could you run "sudo mountall --debug > mountall.log 2>&1" and attach that to this bug - then after you reboot, let me know whether it worked or not.

Thanks

[ooops (ooops)]

Attached is the output before reboot with karmic and ppa mountall. Note
that mountall with karmic mountall does not terminate. I had to stop it
with a ctrl-c. What is attached is the output until that point.

The boot with new mountall itself hangs and cannot be interrupted by
sysreq-e. Thus I was not able to debug. Actually it even hangs in case I
use the /dev/sda5 in fstab. So I have no idea why and where it hangs.
Actually, I see now all the disks in /dev/disk/by-uuid. I have no idea
what changed in the system.

I can do more tests in about a week.

On Thu, 2009-10-08 at 02:15 +0000, Scott James Remnant wrote:
> I've uploaded a new mountall package to the ubuntu-boot PPA:
>
> https://launchpad.net/~ubuntu-boot/+archive/ppa
>
> I would appreciate it if you could install this and try it out.
> *BEFORE* you reboot though, could you run "sudo mountall --debug >
> mountall.log 2>&1" and attach that to this bug - then after you reboot,
> let me know whether it worked or not.
>
> Thanks
>
> ** Changed in: mountall (Ubuntu)
> Status: Triaged => Incomplete
>

[Scott James Remnant (Canonical) (canonical-scott)]

Sorry about the issues with the previous PPA versions, as usual things worked just fine when I tested it in the various rigs I have here - of course it flatly failed when installed on normal systems because I hadn't actually tested that ;)

I've uploaded a new ~boot4 version, this one feels much better (and I'm running it on my laptop now :p)

As before, after installing the package but *before* you reboot, please run with --debug and attach the log to the bug - then after rebooting, let me know how it works out.

Thanks for all your help with testing, this is a big change and it's good to know that it's now working for 95% of people and your help getting it work for the final 5% is greatly appreciated!

[Steve Langasek (vorlon)]

> So, after all, I think the initial bug report is related, as mountall should fail/not hang with invalid UUIDs, but my problem goes one
> step further: the cryptsetup devices are not being setup properly anymore.

Daniel, please open a separate bug report on cryptsetup for your issue.

Unfortunately, I think mountall itself is behaving correctly in your case - you have this cryptsetup device bind-mounted to /opt, and /opt is a FHS filesystem which, if absent at boot, could causes services to fail to start (since packages in /opt may have init scripts in /etc/rc2.d that expect the files to be available). So in the general case, *not* waiting for this filesystem to be present before proceeding with the boot could cause serious boot-time instabilities. (This is distinct from bug #448267, where I argue that we should not be waiting for /srv before entering runlevel 2 because nothing in /srv should be a dependency of the system boot.)

So your underlying problem is that cryptsetup isn't correctly setting up your devices, but that it should be doing so. There's no evidence that this is related to the problem ooops is reporting here.

[ooops (ooops)]

Fortunately, all my partitions in /dev/disks/by-uuid reappeared again.
So I cannot reproduce the initial problem. But you can reproduce the
hanging boot easily by creating a /home mount with a wrong uuid
in /etc/fstab.

I have tested this approach (intentionally giving the wrong uuid of
my /home partition) with mountall versions 0.2.2 as it is in karmic at
the time of writing this and 0.2.2~boot2 as published in ppa. In both
cases the boot hangs and cannot be interrupted, neither with ctrl-c nor
with sysreq-e. Sysreq-e kills udev but not mountall anymore.
Uninstalling cryptsetup so that ctrl-c might work does not help either.
You need either a live-cd or another boot-partition to get the system
running again. Attached is the output of mountall --debug of 0.2.2 with
my correct and incorrect /etc/fstab before the reboot as requested.

[Scott James Remnant (Canonical) (canonical-scott)]

mountall is obviously unable to deal with an fstab with the wrong UUID in it.

The case of showing a message on screen and allowing to cancel is covered by a different bug# (and is already done, pending a usplash bug fix)