cryptsetup init scripts are redundant and can break the boot

Bug #473615 reported by Steve Langasek
104
This bug affects 14 people
Affects Status Importance Assigned to Milestone
cryptsetup (Ubuntu)
Fix Released
Undecided
Unassigned
Karmic
Fix Released
Medium
Unassigned
Lucid
Fix Released
Undecided
Unassigned

Bug Description

Binary package hint: cryptsetup

The cryptsetup package in karmic includes both init scripts and an upstart job. The init scripts should be completely redundant, because the upstart job is written such that it will start all crypted volumes listed in /etc/crypttab that aren't marked with 'noauto'. And there have been reports of regressions in 9.10 as a result of these init scripts being present (reported on bug #430496), due to adverse interaction between the init script (rcS runs with 'console owner') and gdm. Removing the init scripts should resolve these hangs, because the cryptsetup job only requires 'console output', not 'console owner', so doesn't compete with X for control of the VT.

However, this change is not without risk of regression, because the upstart job (and the init scripts as well) will skip over any devices that aren't yet available at the time it's called. The job waits for udevtrigger to finish, but that doesn't guarantee that all disks have been detected, so any disks that arrive later won't be decrypted for use. So both before and after, this is racy... after, the odds of running into problems because of the race are greater.

ProblemType: Bug
Architecture: amd64
Date: Tue Nov 3 04:07:39 2009
DistroRelease: Ubuntu 9.10
Package: cryptsetup 2:1.0.6+20090405.svn49-1ubuntu7
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-14.48+ureadahead2-generic
SourcePackage: cryptsetup
Uname: Linux 2.6.31-14-generic x86_64

Revision history for this message
Steve Langasek (vorlon) wrote :
Revision history for this message
Steve Langasek (vorlon) wrote :

Fixed for lucid with the upload of -1ubuntu8. Changelog:

cryptsetup (2:1.0.6+20090405.svn49-1ubuntu8) lucid; urgency=low

  [ Steve Langasek ]
  * Make the 'start' action of the init script a no-op, this should be
    handled entirely by the upstart job now; and remove any symlinks from
    /etc/rcS.d on upgrade. LP: #473615.

  [ Reinhard Tartler ]
  * Add an apport hook
  * import the blkid and un_blkid from debian, LP: #446517
  * also use this script by default (setting in /etc/default/cryptdisks)

 -- Steve Langasek <email address hidden> Wed, 04 Nov 2009 12:06:47 +0000

Changed in cryptsetup (Ubuntu Lucid):
status: New → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote :

Subscribing ubuntu-sru for the karmic side of this.

Martin, would appreciate some input on this proposed change prior to upload, since it does carry significant risk; an alternative approach would be to also hook the upstart job into a udev rule at the same time, to completely eliminate the raciness.

Changed in cryptsetup (Ubuntu Karmic):
importance: Undecided → Medium
milestone: none → karmic-updates
status: New → In Progress
Revision history for this message
Martin Pitt (pitti) wrote :

Taking some notes while I review this, thinking aloud.

- Code-wise, the init script(s) and upstart job differ in the sense that the ones marked as "noearly" are started after the other ones with the init script, whereas the order can potentially change with the upstart job (which only has INITSTATE="init", not two phases). If I read the upstart jobs correctly, then it could even happen that both the init script and the upstart job run at the same time?

If the order of cryptdisks-enable.conf vs. rcS.conf isn't enforced by some other dependency/magic/coincidence, then it already seems bad to run both. So this speaks in favor of removing the init scripts, even though it means that users might suddenly see a change in the order of which encrypted disks are mounted at startup.

- Boot order: /etc/init.d/cryptdisks LSB header says to start after "raid2 mdadm lvm2 evms". The intention is certainly to catch encrypted disks on RAID/LVM volumes. "raid2" and "lvm2" aren't provided by any init script, "evms" is long gone, so we don't need to care about those. /etc/init.d/mdadm provides "mdadm", but mdadm doesn't have a startup init script (just an udev rule). Thus, the init script dependencies intended by the LSB header have never been implemented by our init.d priorities.

I assume this is what you meant with "So both before and after, this is racy".

The actual bzr change looks correct to me wrt. its intention. Two small nitpicks: changelog has an "ad" typo (meaning "and"), and in the dpkg version the "lt-nl" comparison would suffice (this would fix the corner case where these symlinks were deliberately created manually before installation; very pathological, though).

> alternative approach would be to also hook the upstart job into a udev rule at the same time, to completely eliminate the raciness.

This would mean to run all the parsing, modprobe, dmsetup mknodes, etc. code each time an ecrypted device gets added (which could be on an USB key). Semantically this would probably be correct, since these could be handled in crypttab as well. I'm just not sure about how much overhead this would introduce for a running session.

So I propose we do this small SRU for now, and test it with various common setups (encrypted LVM, encrypted root, encrypted root+home, encrypted home, encrypted swap). In lucid we ought to replace the upstart job with an udev rule based solution and also clean up the scriptery around it (much of it looks obsolete, and we still have gems like "CRYPTDISKS_CHECK=vol_id" in /etc/default/cryptdisks, which is doomed to fail).

Revision history for this message
Steve Langasek (vorlon) wrote :

lvm2 is not provided by an init script, correct; this is handled via udev rules. This is also racy, because there's no guarantee that all the LVM devices will be up when 'udevadm settle' ends.

"So both before and after, this is racy" - I was actually referring to the fact that there's no guarantee *in general* that the underlying volumes are available, either before calling /etc/init.d/rcS or before calling /etc/init/cryptdisks-enable.conf. In both cases, it's possible that the volumes won't become available until after the script has run.

The difference before and after this change is that before, we get three chances to catch the underlying volume. After, we only have one chance. This means we can say with a high degree of certainty that this change will regress /for some users/. One example where it will regress is:

 - /dev/sdh is a USB-attached disk which is slow to power up, and routinely does not register with the kernel until well after the coldplugging stage.
 - /dev/sdh1 is mounted as /home.
 - /dev/sdh2 is the PV for a luks encrypted device, which is listed in /etc/crypttab and configured to be mounted as /mnt/S3kritProj3ct.

Prior to this change, what happens is:

 - /etc/init/udevtrigger.conf finishes; /dev/sdh is not yet available.
 - /etc/init/cryptdisks-enable.conf is run (start on stopped udevtrigger). it exits without doing anything, because /dev/sdh2 is not yet present.
 - /dev/sdh is detected; events are emitted for /dev/sdh, /dev/sdh1, and /dev/sdh2.
 - mountall mounts /dev/sdh1 as /home, and emits the "filesystems" signal
 - /etc/init/rc-sysinit.conf runs, calling /etc/init.d/rcS
 - /etc/init.d/cryptdisks-early runs, and sets up the device, resulting in new udev events being emitted
 - mountall mounts /mnt/S3kritProj3ct.

After the change, /etc/init/rc-sysinit.conf doesn't do anything relevant, and /mnt/S3kritProj3ct is never decrypted or mounted.

The same fundamental race is still present - /dev/sdh2 becoming available, vs. udev cold plugging finshing and /etc/init/cryptdisks-enable.conf being called. But there are cases where the behavior after this change is demonstrably more likely to hit the race condition.

Now, the specific example I came up with here probably didn't work in jaunty either, because we didn't have mountall to wait for /home to become available before telling /etc/init.d/rcS to go. But after all, it's racy - and I'm pretty sure this change will make things worse for some users (and will definitely make things better for others).

Does that clarify?

> This would mean to run all the parsing, modprobe, dmsetup mknodes, etc. code each time
> an ecrypted device gets added >(which could be on an USB key). Semantically this would
> probably be correct, since these could be handled in crypttab as well. I'm just not sure
> about how much overhead this would introduce for a running session.

As Scott has argued, users don't choose disk encryption if their priority is boot performance. And users can always mark devices 'noauto' in /etc/crypttab, if they don't want them autostarted - if they *do* want them autostarted, it would be nice to do so more reliably than we do currently.

Revision history for this message
Martin Pitt (pitti) wrote : Re: [Bug 473615] Re: cryptsetup init scripts are redundant and can break the boot

Steve Langasek [2009-11-05 13:15 -0000]:
> Does that clarify?

It does, thanks.

> > This would mean to run all the parsing, modprobe, dmsetup mknodes, etc. code each time
> > an ecrypted device gets added >(which could be on an USB key). Semantically this would
> > probably be correct, since these could be handled in crypttab as well. I'm just not sure
> > about how much overhead this would introduce for a running session.
>
> As Scott has argued, users don't choose disk encryption if their
> priority is boot performance. And users can always mark devices
> 'noauto' in /etc/crypttab, if they don't want them autostarted - if they
> *do* want them autostarted, it would be nice to do so more reliably than
> we do currently.

I actually meant "plug in an ecrypted USB key while the computer is
fully booted up", since the udev rules would still trigger then.

This is not a trivial problem: the cryptsetup upstart/init scripts
need a console or usplash, and are meant for the "boot" stage
(whatever that means in a fully dynamic hotplug world; but for this
purpose it means: usplash is still running or we have a console). OTOH
the devkit-disks/gvfs integration works only when we have a running X.org
session. We must make sure that udev rules don't call cryptsetup on a
console when we are in an X.org session, since this would collide. In
the best case, the script just detects that it doesn't have a console
and give up early.

Also, can udev rules even start programs which have a console, so that
this would work in the first place?

If not, this could perhaps work as a hackish mitigation for an SRU:

  ACTION=="add", SUBSYSTEM=="block", ID_FS_TYPE=="crypto_LUKS", PROGRAM="pidof usplash", RUN+="wrapper_which_runs_do_start"

This is of course not sufficient at all (you might not use usplash at
all, like on a server), but it seems to cover a safe subset of cases
where we always want the script to run; and with usplash_send we don't
need the calling program to have a console on its own.

However, this would need some care to not have usplash be stopped by
the boot (when X comes up) while the device is being set up. (I don't
know right now how to achieve this).

The longer I think about it, the more confused this seems to me :-(

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Revision history for this message
jkulesa (jkulesa) wrote :

>As Scott has argued, users don't choose disk encryption if their priority is boot performance.
>And users can always mark devices 'noauto' in /etc/crypttab, if they don't want them
>autostarted - if they *do* want them autostarted, it would be nice to do so more reliably
>than we do currently.

Just to emphasize on Karmic server, "more reliably" is really an understatement. It often takes 5 or more boots before the timing is right and the passphrase can be entered w/out hanging the console prior to getting cryptsetup working. Sometimes it boots w/out cryptsetup working - sometimes it hangs without fully booting - and just sometimes you can get the timing right and things will muddle thru and boot up. While it can be made to boot after enough proverbial reset buttom mashing - its really quite broken as is.

I'd suggest that working on the "cold boot" scenario is FAR more pressing - make sure upstart dependancies force X to wait for cryptsetup (upstart is kinda new to me so apologies if I'm misstating things) and then just quietly fail if X is started for example if doing udev hot plug luks volumes. Waiting for a perfect "covers all bases" solution when the current situation is fundamentally broken on boot (from where it was in Jaunty) is something I know I'd at least like to see sooner than later.

Thanks for listening! ;-D

Revision history for this message
Jesse Michael (jesse.michael) wrote :

I think I'm running into this bug also.

I have two encrypted partitions, one on a non-removable drive and one on a firewire drive. GDM started up without waiting for me to enter any passwords, and I now have the following showing up in pstree--

     |-sh---rc---S26cryptdisks-e---S26cryptdisks-e-+-askpass
     | `-cryptsetup
     |-sh---sh-+-askpass
     | `-cryptsetup

Because the boot process is partially stalled, other boot-time init scripts haven't run yet, so other things fail. As an example, you can't use "screen" until the "screen-cleanup" init script runs.

Revision history for this message
Jesse Michael (jesse.michael) wrote :

I should also point out that the encrypted partition on the non-removable drive is currently set to "noauto" in fstab, but it still seems to be stalling the boot process to some degree.

Revision history for this message
Steve Langasek (vorlon) wrote :

On Mon, Nov 30, 2009 at 10:04:28PM -0000, Jesse Michael wrote:
> I should also point out that the encrypted partition on the non-
> removable drive is currently set to "noauto" in fstab, but it still
> seems to be stalling the boot process to some degree.

For karmic, you will want to mark it 'noauto' in /etc/crypttab also.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Revision history for this message
Steve Langasek (vorlon) wrote :

Following discussion at UDS, an SRU has been uploaded to the queue with a more complete fix for this, involving an upstart job that triggers on the udev block-device-added event. There are still some corner cases with risk of regression (documented in other bug reports linked from the changelog), but we no longer need the init script to be assured of not running before the device is available.

Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Accepted cryptsetup into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in cryptsetup (Ubuntu Karmic):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Tomas Rudén (tomas-ruden) wrote :

I have tested the package (I believe, I have version 2:1.0.6+20090405.svn49-1ubuntu7.1). My configuration is

/etc/crypttab:
home /dev/sda6 none luks,tries=5

From /etc/fstab:
/dev/mapper/home /home ext4 defaults,nofail,relatime 0 2

With previous cryptsetup I got prompted for password, but it seemed that more than one process where reading my keystrokes, so I had to try several times before it worked.

Now I don't even get prompted for password. I get the prompt where I can press ESC and get a maintenance shell (without a password prompt, but thats another story). In that shell I can do a "cryptsetup luksOpen" and finally mount /home. After exiting the shell the boot continues.

While the boot requires more steps now, it is much more dependable and I don't have to reboot get my /home mounted.

Revision history for this message
Steve Langasek (vorlon) wrote :

Tomas,

Please check that you have the 'watershed' package installed. There is an updated SRU package, -1ubuntu7.2, that adds this missing dependency.

Revision history for this message
Tomas Rudén (tomas-ruden) wrote :

Thanks, installing 'watershed' made it.

Steve Langasek (vorlon)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cryptsetup - 2:1.0.6+20090405.svn49-1ubuntu7.2

---------------
cryptsetup (2:1.0.6+20090405.svn49-1ubuntu7.2) karmic-proposed; urgency=low

  * Depend on watershed.
  * cryptdisks.functions: do_tmp should mount under /var/run/cryptsetup for
    changing the permissions of the filesystem root, not directly on /tmp,
    since mounting on /tmp a) is racy, b) confuses mountall something fierce.
    LP: #475936.

cryptsetup (2:1.0.6+20090405.svn49-1ubuntu7.1) karmic-proposed; urgency=low

  * debian/cryptdisks.functions:
    - wrap the call to /lib/cryptsetup/askpass with watershed, to make sure
      we only ever have one of these running at a time; otherwise multiple
      invocations could steal each other's input and/or write over each
      other's output
    - new function, crypttab_start_one_disk, to look for the named source
      device in /etc/crypttab (by device name, UUID, or label) and start it
      if configured to do so
  * debian/cryptdisks-udev.upstart: new, additional upstart job run once for
    each block device, using the new crypttab_start_one_disk function,
    triggered by udev; this doesn't eliminate the possibility of a race with
    gdm when the decrypted volume isn't a 'bootwait' mount point (since gdm
    kills usplash), but it does eliminate the race between udev and
    cryptsetup. LP: #454898.
    The other cryptdisks-enable job is still needed as well, to give us the
    second pass needed to cover devices that are decrypted using keys stored
    on other encrypted disks. LP: #443980.
  * debian/cryptdisk.functions: initially create the device under a temporary
    name and rename it only at the end using 'dmsetup rename', to ensure that
    upstart/mountall doesn't see our device before it's ready to go.
    LP: #475936.
  * Make the 'start' action of the init script a no-op, this should be
    handled entirely by the upstart job now; ad remove any symlinks from
    /etc/rcS.d on upgrade. LP: #473615.
 -- Steve Langasek <email address hidden> Tue, 22 Dec 2009 23:29:32 +0000

Changed in cryptsetup (Ubuntu Karmic):
status: Fix Committed → Fix Released
Revision history for this message
Jens Getreu (getreu) wrote :

I found the attached workaround on the internet and everything worked fine until today.

Then there was this apt-get update : .......

Préparation du remplacement de cryptsetup 2:1.0.6+20090405.svn49-1ubuntu7.2 (en
utilisant .../cryptsetup_2%3a1.0.6+20090405.svn49-1ubuntu7.2_amd64.deb) ...
Dépaquetage de la mise à jour de cryptsetup ...
Traitement des actions différées (« triggers ») pour « ureadahead »...
ureadahead will be reprofiled on next reboot
Traitement des actions différées (« triggers ») pour « man-db »...
Paramétrage de cryptsetup (2:1.0.6+20090405.svn49-1ubuntu7.2) ...
update-initramfs: deferring update (trigger activated)
update-rc.d: warning: cryptdisks-early start runlevel arguments (none) do not ma
tch LSB Default-Start values (S)
update-rc.d: warning: cryptdisks start runlevel arguments (none) do not match LS
B Default-Start values (S)

So I rolled back to the original mountall.conf, but the situation is as bad as it was before :
It need serveral reboot attempts to enter the password.

What is wrong?

Revision history for this message
Steve Langasek (vorlon) wrote :

yens,

Please be specific - why do you need several reboot attempts to enter the password? What is happening when you try to enter the password?

Revision history for this message
Chris Olin (chris.olin) wrote :

I appear to have the same problem yens is having, except I'm not prompted for the passphrase at all. Trying to mount the root device times out and I'm dropped into a busybox shell, where I can manually run cryptsetup luksOpen /dev/sda1 crypt1, but it won't continue booting. This problem appeared randomly, starting with a shared library error with libgpg-error0 not being in the initramfs image. Spent forever fixing that, but I still don't get prompted to enter my passphrase.

Preparing to replace cryptsetup 2:1.0.6+20090405.svn49-1ubuntu7.2 (using .../cryptsetup_2%3a1.0.6+20090405.svn49-1ubuntu7.2_i386.deb) ...
Unpacking replacement cryptsetup ...
Processing triggers for ureadahead ...
Processing triggers for man-db ...
Setting up cryptsetup (2:1.0.6+20090405.svn49-1ubuntu7.2) ...
update-initramfs: deferring update (trigger activated)
update-rc.d: warning: cryptdisks-early start runlevel arguments (none) do not match LSB Default-Start values (S)
update-rc.d: warning: cryptdisks start runlevel arguments (none) do not match LSB Default-Start values (S)

Revision history for this message
Chris Olin (chris.olin) wrote :

Actually, I forgot something. Originally, I did get a prompt to enter my passphrase, but it kept coming back as incorrect. When I started trying to fix this problem is when the passphrase prompt went away entirely, and I would get the shared library error if I try manually mounting the cryptsetup command while in the busybox shell.

Dunno if that extra bit of information will help.

Revision history for this message
Steve Langasek (vorlon) wrote :

> Trying to mount the root device times out and I'm dropped into a busybox shell

This bug report is about decrypting disks after the root filesystem is already mounted. Please file a separate bug report for the issue you're seeing.

Also, the library dependencies of cryptsetup are expected to be pulled into the initramfs automatically by update-initramfs. Are you using some other non-standard tool to generate your initramfs?

Revision history for this message
alf (aldanblues-deactivatedaccount) wrote : Re: [Bug 473615] Re: cryptsetup init scripts are redundant and can break the boot

ChrisOlin пишет:
> I appear to have the same problem yens is having, except I'm not
> prompted for the passphrase at all. Trying to mount the root device
> times out and I'm dropped into a busybox shell, where I can manually run
> cryptsetup luksOpen /dev/sda1 crypt1, but it won't continue booting.
> This problem appeared randomly, starting with a shared library error
> with libgpg-error0 not being in the initramfs image. Spent forever
> fixing that, but I still don't get prompted to enter my passphrase.
>
> Preparing to replace cryptsetup 2:1.0.6+20090405.svn49-1ubuntu7.2 (using .../cryptsetup_2%3a1.0.6+20090405.svn49-1ubuntu7.2_i386.deb) ...
> Unpacking replacement cryptsetup ...
> Processing triggers for ureadahead ...
> Processing triggers for man-db ...
> Setting up cryptsetup (2:1.0.6+20090405.svn49-1ubuntu7.2) ...
> update-initramfs: deferring update (trigger activated)
> update-rc.d: warning: cryptdisks-early start runlevel arguments (none) do not match LSB Default-Start values (S)
> update-rc.d: warning: cryptdisks start runlevel arguments (none) do not match LSB Default-Start values (S)
>
>
ТЫ Смотрел фильм Аватар скоро мы к этому придем

Revision history for this message
alf (aldanblues-deactivatedaccount) wrote :

Steve Langasek пишет:
>> Trying to mount the root device times out and I'm dropped into a
>>
> busybox shell
>
> This bug report is about decrypting disks after the root filesystem is
> already mounted. Please file a separate bug report for the issue you're
> seeing.
>
> Also, the library dependencies of cryptsetup are expected to be pulled
> into the initramfs automatically by update-initramfs. Are you using
> some other non-standard tool to generate your initramfs?
>
>
Посмотри фильм Аватар я думаю мы скоро к этому придем.

Revision history for this message
Steve Langasek (vorlon) wrote :

> Посмотри фильм Аватар я думаю мы скоро к этому придем.

This is not a movie discussion forum. Please take this elsewhere.

Revision history for this message
William Wolf (throughnothing) wrote :

The watershed + cryptsetup packages from -proposed fixed the problem for me, but for my /home partition to be decrypted/mounted, itasks me for my password 3 times every time. The root partition only asks once, and then the /home asks me 3 times every time.

I have / encrypted, /home encrypted, and my swap encrypted using a random key, and nothing else except for /boot which is unencrypted, of course. Also, everything is in LVM.

This is way better than how things were before, everything is at least consistent and reliable now. Thanks!

Revision history for this message
William Wolf (throughnothing) wrote :

I guess I need to retract my last post. The first few times, it behaved as I described above, but now it seems to be completely random. Most of the time I can only get in by doing the recovery mode grub option, then trying to type in my password unsuccessfully a few times, dropping to a root shell, mounting the filesystem myself, and then resuming the boot. Anything else is severely unreliable, and it seems it is because the stdin doesnt get to the cryptsetup process looking for the password. It seems that the latest watershed + cryptsetup packages do not solve the problem, in my case at least. I'm not sure what information I can provide to help pinpoint the problem, but let me know if I can be of any further help.

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 473615] Re: cryptsetup init scripts are redundant and can break the boot

On Sun, Jan 24, 2010 at 11:15:49PM -0000, WilliamWolf wrote:
> It seems that the latest watershed + cryptsetup packages do not solve the
> problem, in my case at least. I'm not sure what information I can
> provide to help pinpoint the problem, but let me know if I can be of any
> further help.

Are you using usplash (booting with the 'splash' kernel option)? I think
booting with splash should let this work; we can't realistically fix this
for the no-splash use case in karmic.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Revision history for this message
William Wolf (throughnothing) wrote :

Ahh, you are right. I had gotten used to using the recovery mode kernel instances in grub, but when I do the non-recovery mode (with splash), it does work, but as I noted in my first message, I always have to type my password 3 times. It's as if the first 2 times don't get to cryptsetup properly. It does appear to work reliably with typing my password 3 times, though.

Revision history for this message
Dave (dfluff) wrote :

I am seeing very similar, frustrating behaviour, with a brand new install of 9.10 (amd64), which I then updated meaning I have cryptsetup 2:1.0.6+20090405.svn49-1ubuntu7.2 and mountall 1.0, watershed 4, etc.. No 'essential' mount points are encrypted (which I believe is a factor), only swap and 'data', although 'data' does include, among other things, /home.

If I don't have 'bootwait', gdm starts, but my encrypted partition isn't mounted (and switching back to tty1 there is no sign of a prompt for the passphrase).

If I do set 'bootwait' for the partition, the boot stops when mounting fails. A few hits of return sometimes gives a passphrase prompt, which doesn't work, saying the passphrase is wrong, as if it's only seeing the 'return', and not any other keystrokes. After a mixture of repeated attempts (so cryptsetup fails?) or waiting for mountall to timeout (I'm guessing), I eventually managed to get a maintenance prompt, so that I could remove the 'bootwait', and at least get back in to a state where the machine booted. (trying to boot in to single user mode to remove the 'bootwait' didn't work, because it still hit the failing mount).

Comment #27 suggests it now 'only works with usplash' (which seems a bit dangerous - what if something else causes usplash to not appear/exit early, or if you have some other reason why you don't want it?). I too had become used to booting without 'splash' whilst investigating this - when I tried putting 'splash' back, I got the message about hitting ESC for a recovery console, but *no* keyboard input did anything, the boot had completely hung. Interestingly (worryingly) even my usual Alt+SysRQ+S/U/S/B (sync/unmount/sync/boot) did nothing!

My most recent boot was without splash, with bootwait on my 'data' partition. This time I needed to input the passphrase three times, one time resulted in the passphrase being printed on the console after I'd hit return. If I switch back to tty1, it's still there, in plain sight. It did, however, result in the crypt-partition being unlocked.

/etc/crypttab:
# <target name> <source device> <key file> <options>
data /dev/sda6 none luks
cswap /dev/sda5 /dev/urandom swap

/etc/fstab:
/dev/mapper/data /mnt/data ext3 defaults,relatime 1 2
/dev/mapper/cswap none swap sw 0 0

(sometimes with, sometimes without bootwait on the options)

cswap appears to work *sometimes*, but usually only after a huge delay. It appears to have work this time, after my eventual successful unlocking of 'data', but dmesg for previous boots has shown me that it's come up after between 2000-8000 seconds.

Cheers,
--Dave

Revision history for this message
William Wolf (throughnothing) wrote :

I will say that after some time, my current set up works a lot of the time (using usplash), but I occasionally get the same error that Dave mentions above about usplash telling me to hit "ESC" to drop to a console, but no keyboard input does anything (except ctrl + alt + del still reboots for me every time). It's really frustrating when this happens -- I usually have to reboot several times before it decides to ask me for my /home password and not sit there indefinitely telling me to hit the escape key.

My setup is a little different since I have / encrypted, but / *always* accepts my password on the first try, and *always* mounts properly. The only thing that doesnt want to always mount properly is /home, which is what causes it to give me the "press ESC to drop to a console" when it fails to prompt me for a password for this partition and thus can't mount it.

And of course, when it does properly ask me for my password on the /home partition, i *always* have to type the password in 3 times to get it to accept it. It's as if the first 2 times the input is being directed somewhere else or something....much like it seems to do when I boot without usplash.

When booting without usplash I also get the same symptoms as dave in that the first time I type my password it shows up in plain text on the screen after hitting enter always, and pretty much stays there for the remainder of that boot. I can usually get the machine booted by typing my password a couple of times after that though -- sometimes into a prompt, but sometimes without a prompt even showing.

Revision history for this message
Sven 'Darkman' Michels (sven+launchpad) wrote :
Download full text (3.5 KiB)

Hi,

i've the same issue which is driving me nuts actually.

Setup is Ubuntu 9.10 with /home and swap encrypted (setup since 6.10 or so, can't remeber exactly when i added it). It worked fine till 9.10, then the insane problems started. It sometimes work (e.g. i can enter the passphrase at boot) but mostly i'm unable to boot
my maschine since the boot process just hangs.

Today, the problem even got worse. I updated my box today, which installed a new kernel, so i had to reboot. Interesting thing is, that last time i rebooted (28th Jan iirc) i got prompted for my passphrase at the "gui" (i think its upsplash) and all worked fine. So i thought the Problem is gone. Today, nothing(!) works. In the past, a few reboots made it work from now and then and i "accepted" the workaround. Today, even 20 times of booting made it impossible to enter a passphrase at all. The upsplash only shows that one or more devices (/home in this case) failed to mount and offers me a rescue console when pressing ESC. This console works for, lets say a minute or two, after that, next to 0 keystrokes made it to the console. A few times i saw output from cryptosetup which seems to "get" my rootpassword (when entering the rescue console, it asks for the root password, but the keystrokes seems to get to cryptsetup, but only a few/sometimes...) and thus still fails to boot. If i try to enter my cryptodisk password, i got a failed login, so i don't know when i'm able to enter what ;)

After a few workarounds etc. i found, still nothing worked. The workarounds i tried: bootwait to mountpoint, removed "splash" from the kernelparameters, trying to switch to a non gui runlevel... so the good old /etc/init.d/cryptodisks restart && mount /home gave me back my home (i just have to be quick enough) and with crtl + d the boot process continues normally... and voila,
here i am...

So for me, the new version of cryptsetup seems to made it better in the first (reboot of 28th january worked fine), but something after that made it even worse. The following packages where installed since 28th jan:

hal 0.5.13-1ubuntu8
man-db 2.5.6-2
libfuse2 2.7.4-1.1ubuntu4.3
fuse-utils 2.7.4-1.1ubuntu4.3
libwbclient0 2:3.4.0-3ubuntu5.4
libsmbclient 2:3.4.0-3ubuntu5.4
samba-common 2:3.4.0-3ubuntu5.4
smbclient 2:3.4.0-3ubuntu5.4
samba-common-bin 2:3.4.0-3ubuntu5.4
libc-bin 2.10.1-0ubuntu16
initramfs-tools 0.92bubuntu53
man-db 2.5.6-2
hicolor-icon-theme 0.10-2
desktop-file-utils 0.15-2ubuntu1
menu 2.1.41ubuntu1
libid3-3.8.3c2a 3.8.3-7.2ubuntu2
id3v2 0.1.11-4
vorbis-tools 1.2.0-6
gtkpod-data 0.99.14-2ubuntu3
gtkpod 0.99.14-2ubuntu3
libc-bin 2.10.1-0ubuntu16
menu 2.1.41ubuntu1
man-db 2.5.6-2
hicolor-icon-theme 0.10-2
desktop-file-utils 0.15-2ubuntu1
linux-image-2.6.31-19-generic 2.6.31-19.56
libdevkit-power-gobject1 011-1ubuntu2
devicekit-power 011-1ubuntu2
libgtk2.0-common 2.18.3-1ubuntu2.2
libgtk2.0-0 2.18.3-1ubuntu2.2
libgtk2.0-dev 2.18.3-1ubuntu2.2
libgail18 2.18.3-1ubuntu2.2
libgail-common 2.18.3-1ubuntu2.2
libgtk2.0-0-dbg 2.18.3-1ubuntu2.2
gtk2-engines-pixbuf 2.18.3-1ubuntu2.2
gnome-power-manager 2.28.1-0ubuntu1.3
libgtk2.0-bin 2.18.3-1ubuntu2.2
linux-image-generic 2.6.31.19.32
linux-generic 2.6.31.19.32
...

Read more...

Revision history for this message
Jamie Lokier (jamie-shareable) wrote :

Just another data point. I've been seeing the "asks for your passphrase 3 times" problem ever since I upgraded to 9.10 back in 2009-10, along with the "press ESC to get a root shell" type message, sometimes twice.

I only have /home encrypted.

Although it feels a bit like the passphrase input is being eaten by another program, I've tried various combinations of pressing Enter at the passphrase prompts, thinking that maybe at some step through the process, the line I enter will be seen be a passphrase reader and not another process, but with all the combinations I've tried, it always asks again at least once, and I've never seen any other program do anything interesting in response to my typing. In all cases, the characters typed appear as '*'s, not visible.

I do use usplash, and have to; the boot gets completely stuck for some reason unknown to me quite early if I boot with the text console. (It was fine in 9.04).

I do get occasional random boot lockups, but I've attributed that to KMS, X or some unknown kernel mystery. After all, X locks up every 1-2 weeks anyway (has done since 9.04; Intel 945GM graphics).

Luckily for me, apart from occasional boot hangs, I've not had a problem mounting my /home, after typing the passphrase twice or three times. I haven't noticed any changed in behaviour between 2009-10 and now.

Revision history for this message
fm (fmartens+launchpad) wrote :

Today I deemed it safe to upgrade my ubuntu 9.04 to 9.10. After 5+ months I expected a smooth upgrade.
I had splash disabled. I have /home on a lvm luks volume.
Multiple cryptsetup processes seem to be started, directing output/input to something else than tty1.
With rebooting, trying again, different timings, blind escape and returns and the passphrase itself it is possible to get past the prompt for the passphrase (which is invisible initially).

After reading on launchpad I configured splash. I now have to type my passphrase (which is 32+ chars) 3 times, then it unlocks the /home and boots fine.

This behaviour matches other comments on this bug, if I should open a new bug, please let me know.

Revision history for this message
papukaija (papukaija) wrote :

@fm: This bug is fixed on 2010-01-14. Please open a new bug report with "ubuntu-bug cryptsetup" if you still have problems with cryptsetup.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.