[cryptsetup] library dependency in /sbin/cryptsetup

[roodee]

/sbin/cryptsetup has library dependencies in /usr/lib. If /usr is a separate partition the cryptdisks-early startup fails as the cryptsetup binary cannot find the necessary libraries (/usr has not been mounted yet). This breaks encrypted swap on systems with separate /usr.

[Jani Averbach (jaa-jaa-iki)]

This also affect encrypted swap space severely - the swap setup fails, and you end up to be without swap
(If you don't activate it by your self).

[Reinhard Tartler (siretart)]

jaa: how does this affect swap space? Can you please elaborate on this a bit more?

[Reinhard Tartler (siretart)]

Confirming and reassigning to ijackson. We introduced that change on purpose.

Ian can you please elaborate on this? IIUC, you want to set this bug to Wontfix, I think

[Jani Averbach (jaa-jaa-iki)]

Hello Reinhard,

I am using encrypted swap, and I have following setup:

================
/etc/crypttab:
...
#target device keyfile options
swap /dev/sdaXX /dev/random swap

================
and in /etc/fstab
...
/dev/mapper/swap none swap sw 0 0

================

So I am without swap after boot. Crytpsetup fails when it tries to setup swap, because it links against /usr/lib/libgcrypt.so.11 and /usr/lib/libgpg-error.so.0, and /usr is not mounted at that time.

This is an up-to-date Gutsy Gibbon system.

Best Regards,
Jani

[Reinhard Tartler (siretart)]

yes, this is indeed both convincing and critical,

Ian, breaking crypto swap is really not an option. I think we should rethink the dynamic linking.

[Ian Jackson (ijackson)]

Perhaps a better answer would be to enable the swap later in this case.

[Ian Jackson (ijackson)]

JOOI, what software set up your encrypted swap, or did you follow some recipe somewhere ?

[Jani Averbach (jaa-jaa-iki)]

Hello Ian,

IIRC, the swap is set up by boot scripts and cryptsetup/crypttab, which, I think, are standard parts of debian/ubuntu.
The "recipe" came from man pages:

man crypttab
man cryptsetup

In any case, IMHO, it is not acceptable to link to under /usr from /sbin.

[Ian Jackson (ijackson)]

jaa writes ("[Bug 139635] Re: [cryptsetup] library dependency in /sbin/cryptsetup"):
> IIRC, the swap is set up by boot scripts and cryptsetup/crypttab,
> which, I think, are standard parts of debian/ubuntu. The "recipe"
> came from man pages:
>
> man crypttab
> man cryptsetup

Hmm.

> In any case, IMHO, it is not acceptable to link to under /usr from
> /sbin.

Yes, I agree in general. Unfortunately the situation is rather
complicated here and there isn't an easy fix.

I think that for gutsy I'm just going to have to advise you to adopt a
workaround: I suggest you reconfigure your init scripts so that the
swap setup takes place later.

I hope that in hardy we'll have a cooked answer for encrypted swap,
which will have to involve some kind of `solution' to this problem
(although we can't guarantee that it will support your particular
setup).

Thanks anyway for your bug report and sorry not to be more helpful.

Regards,
Ian.

[Jani Averbach (jaa-jaa-iki)]

Hello Ian,

I have to say I am disappointed about your reply, and I think you are doing the wrong thing here.

Please let me explain:
- You are breaking existing, documented, working feature
- The new, broken behavior is not documented on crypttab's man page
- You are breaking against File Hierarchy Standard by linking /usr/lib from /sbin

I hope you could rethink your solution, for example, remove the new feature, provide two different binaries (/sbin and /usr/sbin), or link it statically. Also, if none of these is not feasible, could you explain a little bit more what is going on, and update the documentation.

Thank you,
Jani

[Sébastien Braun (sebb-yellowhippy)]

Hello all,

I'll just jump in, because I'm lucky to see this today, before I upgrade my feisty systems to gutsy.

This bug has not been active for two weeks. This is a horrible situation for anyone who depends on encrypted storage for whatever reason (company laptop with sensitive data or something), and this is because, as Jani said, this change is breaking, unconditionally, a feature that users are relying on, for no obvious reason. (Where "trust me, there is a reason" does not count as obvious).

I'm more than willing to help in an effort to fix the bug, but this will not be possible for any community member or any other person who does not know the nature of the problem. As it stands now, an upgrade to gutsy simply is not an option for me.

Although, for the record, I saw it work on a system with libgcrypt.so.11 and libgpg-error.so.0 manually copied to /lib (which made me cringe, but a non-working installation makes me cringe even more.)

Please, please, please do something about this situation. At least provide us some information. I don't think leaving everything as it is is acceptable. And, concerning the "workaround" that the grandparent has suggested: This is not something that I'd like to watch some friends of mine do.

Regards,
Sebastien

[Swâmi Petaramesh (swami-petaramesh)]

I was just hit by this one in Gutsy :-( Sh**, my encrypted partitions fail mounting (and my system fails starting) just because libraries cryptsetup depends upon lie in /usr/lib where they should be in lib :-(

[Swâmi Petaramesh (swami-petaramesh)]

And by the way, the links "solution" is definitely not one. In my case, /usr resides in a separate partition from the rootfs, so having cryptsetup depending upon anything which is under /usr is a *bad idea* because cryptsetup is called before /usr is mounted...

Futhermore, it's not just an encrypted swap issue, but an issure for any encrypted filesystem. It hits my /tmp and crashes my system boot :-(

Libraries on which cryptsetup depends should go into /lib and not /usr/lib, period.

[Jens Gustedt (jens-gustedt)]

Yet to report another hit of this bug. Took me some hours to find out why my swap wouldn't come at boot time, altough I followed all the documentation.
I am really surprised to find that cryptsetup depends on dynamic libraries. Not only the ones in /usr/lib, but in general it is a bad idea. This executable is critical for the early boot phase, Messing around with it is playing with fire.

Found a workaround, which is to list /usr as a dependency in crypdisks, but this only works since I only have swap encrypted for the moment. Thinking of what have happend if I'd just naively had encrypted my root partition. Undermines my confidence in Ubuntu.

[Swâmi Petaramesh (swami-petaramesh)]

I could temporarily work around this by manually copying libgcrypt.so.11.2.3 and libgpg-error.so.0.3.0 to /lib, with their symlinks, but this is not a satisfactory solution.

I'm dissatisfied seeing that this bug importance is yet set to "medium" as this can prevent a system from booting ; I feel that solving this should be high priority so it doesn't hit too many people in the face (those who haven't upgraded Feisty => Gutsy yet but may soon do...). IMHO, this bug is critical.

[Jens Gustedt (jens-gustedt)]

>>>>> ``Swâmi'' == Swâmi Petaramesh <email address hidden> writes:

    Swâmi> I could temporarily work around this by manually copying
    Swâmi> libgcrypt.so.11.2.3 and libgpg-error.so.0.3.0 to /lib, with
    Swâmi> their symlinks, but this is not a satisfactory solution.

I have now gone even further, I copied the statically linked
executable from http://luks.endorphin.org/dm-crypt and started to use
that, now.

    Swâmi> I'm dissatisfied seeing that this bug importance is yet set
    Swâmi> to "medium" as this can prevent a system from booting ; I
    Swâmi> feel that solving this should be high priority so it
    Swâmi> doesn't hit too many people in the face (those who haven't
    Swâmi> upgraded Feisty => Gutsy yet but may soon do...). IMHO,
    Swâmi> this bug is critical.

I completely agree, this *is* critical. To say that loud and clear, if
you are currently in Feisty and have encrypted partitions for which
you need cryptsetup,

    DON'T UPGRADE TO GUTSY

unless you think you master this bug. In particular, keep a live cd
arround.

Jens

[Swâmi Petaramesh (swami-petaramesh)]

I agree that static binaries are *much* better for critical binaries that are needed to mount filesystems. But if the Ubuntu team wants to keep cryptsetup dynamically linked, wouldn't there be an easy solution modifying the libgcrypt11 and libgpg-error0 so they simply install into /lib instead of /usr/lib ?

[Jens Gustedt (jens-gustedt)]

Hello,

>>>>> ``Swâmi'' == Swâmi Petaramesh <email address hidden> writes:

    Swâmi> I agree that static binaries are *much* better for critical
    Swâmi> binaries that are needed to mount filesystems. But if the
    Swâmi> Ubuntu team wants to keep cryptsetup dynamically linked,
    Swâmi> wouldn't there be an easy solution modifying the
    Swâmi> libgcrypt11 and libgpg-error0 so they simply install into
    Swâmi> /lib instead of /usr/lib ?

They probably have their reason why they think these belong in
/usr/lib.

Another would perhaps be an install option, to link against static
version of the libraries if they are installed. But I am quite new to
Ubuntu, so I can't see how easy this is.

But in any case I don't have the impression that much people are too
much worried about this bug. First it was declassified and now it is
not even assigned to anybody anymore. To be honest, when I switched to
Ubuntu recently, I expected a bit more.

Jens

[Stephen Howard (sjhoward)]

I've spent the last four days trying to figure out why my previously working (in feisty) cryptdisks were silently failing to mount during gutsy's boot, but would happily mount manually if I invoked the scripts after login.

This is a critical bug because:
1. anyone who *properly* uses Ubuntu's *standard* encryption init.d scripts to mount an encrypted /tmp will leave their Ubuntu system unbootable - this is the case even though the user did everything right. Any bug that leaves a system unbootable must be rated critical.
2. encrypted swap fails to mount, *and fails silently* so there are people out there with laptops relying on Ubuntu's encryption infrastructure who are going to be a bit upset;
3. cryptdisks is normal init.d infrastructure - people who use it are not doing something weird.
4. linking against a mount point that doesn't necessarily exist at the Sxx time cryptdisks is executed is plain silly.

[Reinhard Tartler (siretart)]

I'm on this. Will fix it by installing the libgpg-error and libgcrypt11 libraries in / instead of /usr

[Reinhard Tartler (siretart)]

libgcrypt11 (1.2.4-2ubuntu3) hardy; urgency=low

  * move library from /usr/lib to /lib (LP: #139635)

 -- Reinhard Tartler <email address hidden> Tue, 30 Oct 2007 11:15:02 -0400

[Swâmi Petaramesh (swami-petaramesh)]

Good news there will be a fix ! Any idea about the time it will take to reach the Gutsy updates ?

Besides this, I'm rather surprised to see the previous comment display "Urgency = LOW" when I otherwise see that listing the "release critical" bugs for Gutsy, importance "high" displays in the top 6 things such as :

- Azureus hangs or crashes showing splash screen at start
- Missing package dependencies in bittorrent-gui

(Too bad, kids wont' be able to play with P2P with no trouble...)

But "encrypted filesystems cannot mount and system boot fails" gets a "urgency = low" ??

No kidding ?

Makes me wonder if I was not mistaken somewhere... Anybody else got some kind of the same feeling ?

[Stephen Howard (sjhoward)]

Thanks Reinhard.

[Reinhard Tartler (siretart)]

working on it per discussion with Martin Pitt

[Swâmi Petaramesh (swami-petaramesh)]

I added "libgpg-error" to the "affects" as cryptsetup has dependancies to both libgcrypt11 and libgpg-error0, both currently being in /usr/lib, and both needed to be moved to /lib ...

[Fabien Tassin (fta)]

This is causing me troubles (in Hardy) with libtool while compiling some Gnome apps like rhythmbox.

ar cru .libs/libaudioscrobbler.a rb-audioscrobbler-plugin.o rb-audioscrobbler.o rb-lastfm-gst-src.o rb-lastfm-source.o
ranlib .libs/libaudioscrobbler.a
creating libaudioscrobbler.la
/bin/sed: can't read /usr/lib/libgcrypt.la: No such file or directory
libtool: link: `/usr/lib/libgcrypt.la' is not a valid libtool archive
make: *** [libaudioscrobbler.la] Error 1

digging deeper into libtool shows it's caused by libgnutls13 (1.6.3-1build1) still referring to /usr/lib

[Martin Pitt (pitti)]

Reinhard, seems it needs to be fixed with --prefix after all (not just with dh_install), so that the .la files become correct, too?

[Martin Pitt (pitti)]

Ah, it might just be because gnutls13 does not get built in Hardy, it's depwaiting on some universe package.

[Martin Pitt (pitti)]

I sorted it out, gnutls13 should build in hardy soon.

[Fabien Tassin (fta)]

Here is a patch for libgpg-error-1.4 fixing the /usr/lib -> /lib migration.

[Fabien Tassin (fta)]

another patch for libgcrypt11-1.2.4

[Fabien Tassin (fta)]

gnutls13 is still broken as libtool seems to expect all dependencies in /usr/lib.
no patch (yet?), sorry.

[Cesare Tirabassi (norsetto)]

I can confirm the same problem with claws-mail which ftbfs:

libtool: link: cannot find the library `/usr/lib/libgcrypt.la' or unhandled argument `/usr/lib/libgcrypt.la'
make[6]: *** [libclawsgtk.la] Error 1

[Fabien Tassin (fta)]

ok, i got it fixed locally. I just forgot opencdk10 yesterday.

1/ apply libgpg-error-1.4-fix_lib_path.patch and rebuild libgpg-error
2/ apply libgcrypt11-1.2.4-fix_lib_path.patch and rebuilt libgcrypt11 with the new libgpg-error
3/ rebuild libopencdk10 (untouched) with the new libgcrypt11
4/ now you can rebuild gnutls13 (untouched) with the new libgcrypt11 & libopencdk10

[Martin Pitt (pitti)]

I fixed libgpg-error:

 libgpg-error (1.4-2ubuntu3) gutsy; urgency=low
 .
   * Configure with libdir=/lib, so that reverse build dependencies don't look
     for the library in /usr/lib because the .la says so. (see LP #139635)
   * Do not install the .la file, we want to get rid of them in the long term
     anyway.

Steve Kowalik is currently handling libgcrypt11, which needs a similar fix.

[Cesare Tirabassi (norsetto)]

Well, I'm still getting this ftbfs error:

libtool: link: cannot find the library `/usr/lib/libgpg-error.la' or unhandled argument `/usr/lib/libgpg-error.la'

This is a sparc build:

http://launchpadlibrarian.net/10290664/buildlog_ubuntu-hardy-sparc.claws-mail_3.0.2-1_FAILEDTOBUILD.txt.gz

which is using 1.4-2ubuntu5 (libtool 1 - Stevenk 0 :-))

[Martin Pitt (pitti)]

Please try again now with current Hardy, it should be fixed now.

[Cesare Tirabassi (norsetto)]

Yes, it compiles with no problem.

[ad (aratod)]

Hi!

First I just would like to enforce the importance of not to use the /usr/lib during boot. I had a several days war with gutsy and the encrypted /usr on a separated partition. But thanks to google and launchpad now I can see the workaround solutions (eg. copy the two libs into /lib) but I do not like that.

One more thing is, what if the /var is on a separated encrypted partition, too? I was not able to boot because of this problem, too. I do not know if this is the right place to inform (I am afraid not, but in this case please help me to find it with a proper answer), but during boot, system needs the /var/run/klogd/kmsg as well. But this results the same problem as the encrypted /usr on a separated partition. System wants to use this "file" before /var is mounted. :-( (I still try on to find some google answers in this topic, as well.)

D.

[Martin Pitt (pitti)]

Hi ad,

ad [2007-11-19 16:24 -0000]:
> First I just would like to enforce the importance of not to use the
> /usr/lib during boot.

This is already fixed in Hardy, and will eventually get fixed in
7.10 (Gutsy), too.

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntu.com
Debian Developer http://www.debian.org

[Martin Pitt (pitti)]

Not a problem in cryptsetup itself.

[Martin Pitt (pitti)]

No action here for some time, unsubscribing ubuntu-sru. Reinhard, if you want to fix this, please go ahead and upload to -proposed.

[Stephan Müller (megandy)]

I'm not sure wheter it's the same problem but after an update to cryptsetup_1.0.5-ubuntu2.1 my order of mounting cryptodisks especially the swap changed.

old starting order:
mounting of encrypted swap
booting
mounting of encrypted home

current starting order:
booting
mounting of encrypted swap
mounting of encrypted home

more details are in my the post:
http://ubuntuforums.org/showthread.php?t=676822

The problem is that even a rollback to the old version does not change the mounting order. My swap is now unusable because of a prior hibernate. Is there any solution for this problem?

[Reinhard Tartler (siretart)]

sorry, I didn't find the time to work on this. If someone else wants to jump in, please assign this bug to you!

[Sergio Zanchetta (primes2h)]

The 18 month support period for Gutsy Gibbon 7.10 has reached its end of life -
http://www.ubuntu.com/news/ubuntu-7.10-eol . As a result, we are closing the
Gutsy task.