mountall fails to mount encrypted /tmp and swap

[Andreas Heinlein]

Binary package hint: mountall

I have a problem which seems related, but not identical to #52766 and #561390.

I have a lucid test installation with encrypted swap and /tmp

The boot process hangs with the following message:
The disk drive for /tmp is not ready yet or not present.
Continue to wait; or Press S to skip mount or M for manual recover

I can press M, enter root password and then "mount /tmp", Ctrl+D and the system boots up normally.

This problem occurs every time (not only sometimes, as the other bugs describe). I also need to turn on swap manually, though nothing complains about that during booting.

swap and /tmp are set up using /etc/crypttab like this:
crypt_dev_sda5 /dev/sda5 /dev/urandom swap
crypt_dev_sda6 /dev/sda6 /dev/urandom tmp

/etc/fstab contains:
/dev/mapper/crypt_dev_sda6 /tmp ext2 defaults 0 2
/dev/mapper/crypt_dev_sda5 none swap defaults 0 0
---
Architecture: i386
DistroRelease: Ubuntu 10.04
Package: mountall 2.14
PackageArchitecture: i386
ProcEnviron:
 LANG=de_DE.utf8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.32-21.32-generic 2.6.32.11+drm33.2
Tags: lucid
Uname: Linux 2.6.32-21-generic i686
UserGroups: Domain Admins Domain Users GOsa Administrators admin audio cdrom floppy lpadmin plugdev scanner video

[Scott James Remnant (Canonical) (canonical-scott)]

Thanks for the report,

We do have a couple of similar bugs doing the rounds; could you help collect a bit more information to debug this problem?

First run "apport-collect 571682", this will attach various files from your system to the bug which I need to look at.

Next edit /etc/init/mountall.conf, find the "exec" line and add --debug to the end. After rebooting (and skipping those filesystems), attach /var/log/boot.log

Thanks

[Andreas Heinlein (aheinlein)]

apport information

[Andreas Heinlein (aheinlein)]

Here you go...

I have to add that "always fails" is only true for /tmp, swap sometimes (!) works, as it did this time when I created the boot.log.

[Scott James Remnant (Canonical) (canonical-scott)]

Thanks, the important bit from that log is:

try_udev_device: ignored /dev/mapper/crypt_dev_sda6 (not yet ready?)
Skipping /tmp at user request
skip_mount: /tmp

Could you run "udevadm info --export-db" and attach the output for me?

[Vreixo Formoso (metalpain2002)]

I'm having the same issue with a Lucid fresh install. I think this is the same as other bug reports such as bug #475936. Not so sure, however, that it always occurs. I think it is just a matter of luck, as it worked once for me (in more than 20 times). It seems a race condition between the process that handles crypttab and mountall.

[Andreas Heinlein (aheinlein)]

Yes, this is some kind of race condition. Scott, I am not sure if you are aware that in the above setup, the volumes are initialized each time with a random key, and the filesystem (tmp) / swapspace signature is recreated each time on boot. It seems mountall is not prepared for that and waits for the device to become ready, but not for the filesystem/swap signature to be created. If I change the setup to a fixed-keyfile one, where the filesystem is not recreated each time, everything works.

This is a kind of race condition, as it occurs only sometimes on faster (dual-core) machines, but nearly always on slower (single-core) machines. Obviously, on the faster machine, the crypttab handling process manages to get everything done before mountall tries to mount, on the slower one it does not.

[papukaija (papukaija)]

The requested information is provided.

[papukaija (papukaija)]

@Vreixo: That bug is already fixed.

[Mike Buckley (michael-buckley)]

It was not fixed for me as of 6/18/10 with the latest updates.
The following script seems to fix the tmp issue for me. It is a workaround though and not a fix. I've rebooted 10 times in a row and it is still mounting /dev/mapper/crypttmp on /tmp without hanging up the system.
I changed the stock /etc/init/mountall.conf script by commenting out the emits filesystem line. This is on Ubuntu 10.04 LTS.
My custom script called cryptdisks-tmp.conf, placed under /etc/init, has the following lines:
# cryptdisks-tmp - setup encrypted /tmp/device
#
# This is designed simply to setup and encrypted tmp device under
# /dev/mapper called crypttmp and then mount it on tmp.
# This requires a noauto entry in the fstab for the /tmp mount

# Also no crypttab entry is required to setup /dev/mapper/crypttmp
# as all that is taken care of in this script.
# Basically I commented out the "emit filesystem" section in
# /etc/init/mountall.conf, then created this script.
# This script will start once mountall is started and it will emit
# the filesystem emitter
# This will cause most other upstart jobs, which look for the filesystem
# emmitter not to start until this script is done.
# So I think it may fix the /tmp issue
# Also since the code is in a script, I think that the script must finish
# before the filesystem event is emitted. Someone please correct me if I
# am wrong about this.
#
description "script to setup encrypted /tmp to try to bypass race condition"

start on started mountall
emits filesystem
console output
task

script
  if [ -e /dev/mapper/crypttmp ]; then
    echo "For some reason /dev/mapper/crypttmp exists. Bailing out."
    exit 1
  fi
  if mount | grep /tmp > /dev/null 2>&1; then
    echo "/tmp directory is already mounted. Bailing out."
    exit 1
  fi
  ## Now we can proceed to create the /dev/mapper/crypttmp device and mount it.
  if ! cryptsetup create crypttmp /dev/sda3 --key-file=/dev/urandom > /dev/null 2>&1; then
    echo "Failed to create /dev/mapper/crypttmp on /dev/sda3. Bailing out."
    exit 1
  fi
  # Now we need to make the ext2 filesystem on /dev/mapper/crypttmp
  if ! mkfs.ext2 /dev/mapper/crypttmp > /dev/null 2>&1; then
    echo "Create ext2 filesystem on /dev/mapper/crypttmp failed. Bailing out."
    exit 1
  fi
  # We have an fstab entry with noauto in it, so mount /dev/mapper/crypttmp
  if ! mount /dev/mapper/crypttmp > /dev/null 2>&1; then
    echo "Failed to mount /dev/mapper/crypttmp on /tmp. Bailing out."
    exit 1
  else
    echo "/dev/mapper/crypttmp is mounted"
  fi
end script

[bastafidli (ubuntu-bastafidli)]

I see the same problem in maverick but with the following twist.
My root partition is on 2 disks using RAID 1 using mdraid and is encrypted using LUKS. Each of these disks contains also swap partition which is encrypted but not configured using mdraid since kernel doesn't need it.

In my /etc/fstab I have

/dev/mapper/sda2_crypt none swap sw 0 0
/dev/mapper/sdb2_crypt none swap sw 0 0

Only of these partitions is mounted upon start. I do see message

The disk drive... is not ready yet or not present.
Continue to wait; or Press S to skip mount or M for manual recover

[bastafidli (ubuntu-bastafidli)]

I just verified this bug is still present on fully updated maverick but it is a timing issue. I have following encrypted partitions

/dev/mapper/md1_crypt on / type
/dev/mapper/md2_crypt on /home
/dev/mapper/sda2_crypt none swap sw 0 0
/dev/mapper/sdb2_crypt none swap sw 0 0

plymouth asks me for passphrases in following order

1. md1_crypt
2. sda2_crypt
3. sdb2_crypt
4. md2_crypt

If I do not enter password for #3 fast enough, I get message that home is not present and even though when I enter password for it looks like is #3 it gets applied to #4. Then /home directory is mounted and the second swap partition is never decrypted and mounted.

I also had to apply following workaround to get the plymouth display correctly on my graphics card
https://bugs.launchpad.net/ubuntu/+source/plymouth/+bug/563878

[Lex Ross (lross)]

The bug is still present in Oneiric 11.10
And if I only have my swap partition encrypted and not the /tmp the boot process takes way too long with the final message of

The disk drive for swap is not ready yet or not present.
Continue to wait; or Press S to skip mount or M for manual recover

The boot process then continues regardless of whether or not I press S or M key.
And when the system is fully booted, the swap partition is listed as properly activated.