Buggy, under-maintained, not fit for main anymore; alternatives exist

[Dimitri John Ledkov]

The preffered way to get encrypted disks, these days, is to do full disk encryption using LUKS.

If per-directory encryption is required, it is recommended to use fscrypt support in e.g. ext4.

Support for installing using ecryptfs encrypted /home has been disabled in the installers.

Please demote ecryptfs-utils to universe for bionic+

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dimitri John Ledkov (xnox)]

It is in universe now, marking as fixed released.

[Xavier Gnata (xavier-gnata-gmail)]

Ok but what am I supposed to do on a dual boot machine??
I cannot use full disk encryption as I have other partitions for the other OS (or can I??)

fscrypt ? where is it documented??

[Seth Arnold (seth-arnold)]

"Full disk encryption" doesn't actually mean the full disk -- it means a specific block device, so you could leave your NTFS partitions alone and encrypt only your ext4 partitions (and swap partitions, if you use them).

You've got two real options:

- block device encryption such as via dm-crypt or LUKS
- fscrypt front-end to filesystem-native encryption

Probably a bug report is the wrong place to learn about the alternatives. They serve different threat models and interact with the user in different ways, and one or the other may be better integrated into other tooling.

Thanks

[airone (airone)]

fscrypt is not yet ready. As of now, ubuntu 18.04 "LTS" does not offer encryption support for directories. Several use cases cannot be satisfied with full-partition encryption.

https://askubuntu.com/a/1031509

[Linus Torvalds' gracious mode on] Quit the corpo-crap. This is a loss in functionality. Acknowledge it, maybe do nothing about it, but don't lie telling that alternatives exist if you can't support them.

[Jeremy Bícha (jbicha)]

> Acknowledge it

It is acknowledged. That's why it's mentioned in the Release Notes.

[David (dave400)]

No way to encrypt Ubuntu AND swap file when dual-booting with Windows?

[Dimitri John Ledkov (xnox)]

On 24 May 2018 at 22:26, David <email address hidden> wrote:
> No way to encrypt Ubuntu AND swap file when dual-booting with Windows?
>

Yes there is. Use dm-crypt / LUKS. for the partitions that are used by Ubuntu.

[Dmitry Savvateev (savvdm)]

> Yes there is. Use dm-crypt / LUKS. for the partitions that are used by Ubuntu.

Is it possible to do this automatically from (alternative) installer? Or should I learn to do this manually? Full disk encryption is not an option for me. I can only encrypt individual partitions.

Consider also this comment: https://askubuntu.com/questions/1029249/how-to-encrypt-home-on-ubuntu-18-04/1031509#comment1673761_1029249
It should be mentioned clearly in release notes that you can't even log into your encryptfs-encrypted home with Ubuntu 18.04.

[Seth Arnold (seth-arnold)]

On Wed, Jun 06, 2018 at 11:30:59AM -0000, Dmitry Savvateev wrote:
> It should be mentioned clearly in release notes that you can't even log
> into your encryptfs-encrypted home with Ubuntu 18.04.

Dmitry, if do-release-upgrade uninstalled the ecryptfs-utils package
when upgrading, please file a bug against do-release-upgrade.

Thanks

[Redsandro (redsandro)]

@Seth-Arnold I don't think your comment is relevant to the problem. Let expand on @savvdm Dmitry's comment, for I feel that a couple of balls were dropped, too.

Some of us have caught the habit of using a separate /home partition from the early days for the convenience of data-, settings-, and project retention. We have learned - and do so for years - that we can do a fresh install of a Ubuntu, and re-use the /home partition. When /home encryption was introduced, all we needed to do on a fresh install is use the same password, and things would be fine.

Now, without a clear warning, without a working alternative, and without an automatic conversion option, doing what we always did results in a broken/unusuable/reset home.

There's some mention of some deprecation in the release notes for the people who read those, "but surely the installer will take care of the necessary steps involved in transcoding this once promoted and defaulted feature, no?" (Spoiler: No, it does not.)

It should indeed be mentioned clearly in release notes (and in the installer imo) that you can't even log into your encryptfs-encrypted home with Ubuntu 18.04. Otherwise you'd expect something like this in the installer:

"We've detected an unsupported home encryption for user foobar. Would you like to:"

* Remove your prevous home (this will remove all your files);
* Leave your old files in the `.ecryptfs directory` for recovery purposes;
* Decrypt your previous home (all multiboot distros can use this user);
* Pick a different username (other multiboot distros can keep using this user);

[Urop (urop)]

I currently have several installs with encrypted homes. Today I was going to upgrade the first of them, in this case, from 17.10 to 18.04. Having one last scan of the release notes before upgrading, I spotted this issue (https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes#Other_base_system_changes_since_16.04_LTS, 4th bullet). The text there makes it sound like you can no-longer setup encrypted homes using the installer... but that existing encrypted homes should continue to work fine. To be sure, I clicked through to this bug report, and read the summary. Again, there is nothing there to give any indication that existing encrypted home directory setups will be affected adversely. The following surely applies to setting up new encrypted home directories, not to upgrading setups with encrypted homes to new Ubuntu versions? "Support for installing using ecryptfs encrypted /home has been disabled in the installers." But then I read through to comment 11, and now I'm not sure.

I need certainty before I upgrade, so some clarification would be greatly appreciated.

If I upgrade a pre-18.04 system that has an existing encrypted home directory setup, will it continue to just work after upgrading, or will it be broken in some way? If the latter, what exactly needs to be done to fix this? (Also, if the latter, why on earth would you take a deliberate decision to break the systems of people using an advertised feature for years without providing a suitable upgrade path and without even properly advertising it?)

Thank you.

[Chris Rainey (ckrzen)]

It _WILL_ work. I've done it.

17.10 --> 18.04 _upgraded_ with ecryptfs 'home'.

Tools remain installed and unaffected.

As always: BACKUPS-FIRST !! BACKUPS-FIRST !! BACKUPS-FIRST !!

[Urop (urop)]

Thank you Chris. Having first checked my backup, your post gave me the confidence to start the upgrade. It was a complete nightmare. I thought my system was completely borked. The upgrade (using do-release-upgrade) froze part way through, immediately after reporting the following to the console:

Generating a new Secure Boot signing key:
Generating a 2048 bit RSA private key
....+++
..................................................................+++
writing new private key to '/var/lib/shim-signed/mok/MOK.priv'

Waited about an hour, but it was clear that it was stuck and it didn't appear than anything of significance was going on. Finally, I was forced to CTRL-C the upgrade and hope for the best. The upgrade wasn't able to restart itself because it thought something was using apt. So, I had to manually kill various processes that had locked apt, including the bionic process (which was presumably the upgrade process itself?), and was then able to get the upgrade going again... but somehow it left my system with broken dependencies.

Also the upgrade was producing extremely confusing messages such as:

The software on this computer is up to date.
There are no upgrades available for your system. The upgrade will now be cancelled.
Do you want to start the upgrade?
Continue [yN] Details[d]
> d
No longer supported: .... ecryptfs-utils ...

What does that mean? It's completed the upgrade, and it says the software is up-to-date (great!), but that is just after it has told me that there are broken dependencies that can't be resolved... So, when it says the upgrade will be cancelled, does this mean it will try to roll back the changes to recover... and what does it mean by 'starting' the upgrade, when it should have finished already... and what do the responses y and n mean in this context ... and when it says that ecryptfs-utils is no longer supported, so what? Is it planning to remove it, which I don't want, or is it just a warning? It's completely opaque, and ultimately you just have to choose y or n at random and hope for the best, and then go around the loop again when the first thing you tried results in you being presented with the same prompt after something may or may not have changed...

I was finally able to complete the upgrade, but then had to manually resolve the broken package dependencies. There appeared to be multiple problems, but I fortunately tackled what appears to have been the root cause first, which was gir1.2-freedesktop. Neither

dpkg --configure -a

nor

apt-get -f install

worked, so I had to force remove it using

sudo dpkg --remove --force-remove-reinstreq gir1.2-freedesktop

Then I reinstalled it and the packages that were removed with it to recover. I was finally able to boot into an upgraded system, enter my secure boot password and ...

it worked, *including the encrypted home*. How about that for a seamless upgrade! That was from an Ubuntu 17.10 server install, on which xubuntu-core had been installed. Let's hope the other upgrades go a bit more smoothly than that!

[Chris Rainey (ckrzen)]

Good work Urop and great follow-on documentation of your experiences!

Trial by fire, my friend. Truly. You have been baptized and reborn. >:-]

I hope your next upgrade(s) go smoother.

“Was mich nicht umbringt macht mich stärker." ~Friedrich Nietzsche
["what does not kill me makes me stronger."]

[Seth Arnold (seth-arnold)]

On Tue, Jun 12, 2018 at 06:35:56PM -0000, Urop wrote:
> it worked, *including the encrypted home*. How about that for a seamless
> upgrade! That was from an Ubuntu 17.10 server install, on which xubuntu-
> core had been installed. Let's hope the other upgrades go a bit more
> smoothly than that!

I strongly recommend waiting on further upgrades until this python issue
is sorted out:

https://lists.debian.org/debian-devel/2018/06/msg00138.html
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1768379
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/1763844

I don't know why some people hit it and other people do not. It is not
a fun one to hit. Even with a full local archive mirror and decades of
experience with Debian and Ubuntu it took me several hours to repair.

Urop, if you hit something similar but not python, please file a bug.
It'd be nice to sort out as many of these upgrade issues before 18.04.1
as we can. (If you hit py3clean / py3compile issues then a fix is probably
already in progress.)

Thanks

[Paddy Landau (paddy-landau)]

Would people please note that full-disk encryption is already possible, where everything on a new installation is encrypted even including /boot, but without wiping any existing OS such as Windows. (This is why the existing full-disk encryption is not useful for the majority of users.)

This would make a far better option than asking users to manually install fscrypt.

There is a bug with Grub that prevents wide scale adoption of this, for which I have raised a bug:

https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1773457

Reference for full-disk encryption without wiping any other OS:

https://help.ubuntu.com/community/ManualFullSystemEncryption

Thank you

[Urop (urop)]

I'd like to add another use case requiring non-full disk encryption. Machines that need to be powered on and off remotely, which is currently done using a wake-on-lan. The machines are not full-disk encrypted, but make use of encrypted homes and certain encrypted disks/partitions that can be decrypted after boot. With this setup, if security updates or a power cut require a machine to be shut down, it can be rebooted remotely by the owner.

Switching to full disk encryption would be preferable. However, there are currently no simple solutions for remote entry of the system password by the owner of the machine. (There's setup of a separate low power device that can be logged into remotely and that could imitate a USB keyboard to enable remote entry of the system password... but it seems overly complicated and would be yet another thing to maintain.)

A case in point is my own home machine. I spend several months a year out the country, and don't need or want it on all the time while I'm abroad, but need to be able to power it on and access it occasionally.

Unless or until there a simple solution for remote system password entry, non-full disk encryption is still needed, and so it was a mistake to remove this functionality.

[Dimitri John Ledkov (xnox)]

@urop

You should switch to using ecryptfs / ext4 encryption.

For remote password entry, one can include ssh server in the initramfs and use e.g. dropbear i believe it is called, to ssh into the initramfs to enter passphrases to unlock the fully disk encrypted systems.

Either of these things are better supported in the kernel with less bugs that cause data loss.

[Urop (urop)]

Thank you Dimitri. The dropbear initramfs technique looks interesting. It ought to provide a solution for my use case... I'll give it a go.

[RoestVrijStaal (roestvrijstaal)]

Thanks for not including properly documented migration paths from ecryptfs-encryption to encrytion X.

I'll stick to 17.10 for the time being.

[Wes (wesinator)]

@xavier-gnata @dave400 @xnox Bug for supporting encrypted LVM option on dual-boot installation - https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1514120

[Zythyr (zythyr)]

I have similar use-case as @urop. I had my /home partition encrypted using dm-crypt while all other partition remained decrypted. After performing "wake-on-lan", I would remotely login using SSH and manually decrypt the /home partition. I am still a novice user so I coulnd't figure out the dropbear initramfs method. One drawback with my approach was that some services wouldn't start after mnanual decryption using remote SSH. Here is an example of the issue: https://askubuntu.com/questions/1013627/how-to-automatically-start-a-service-after-manually-decrypting-home-partition

[Forest (foresto)]

Another ecryptfs use case that I didn't notice in these comments:

Protecting a directory tree within a user's home directory, to be unlocked for short term use and then re-locked immediately afterward, without logging out or requiring root access. This is appropriate for limiting the exposure of your sensitive files while using software that runs as you (and therefore has access to all your files) but you don't trust to be free of exploits (e.g. web browsers or games).

A common pattern is to exit all programs that don't need access to your encrypted directory, then unlock it and do your viewing/editing, then re-lock it before using complex or proprietary software again. In the physical world, this is like putting your private papers in a locked filing cabinet while guests visit, rather than leaving them on your desk.

LUKS/dm-crypt are not well-suited for this use case, since they require carving out a fixed-size chunk of disk space (which wastes space until it is filled and denies additional storage once it is filled), and since they require root access to set up.

It looks like fscrypt might one day be well-suited for this use case, but it doesn't appear to be ready yet.

That means that Ubuntu does not yet have a good replacement for ecryptfs, which was an officially encouraged tool not very long ago. I hope we'll all keep this in mind before telling people they should not be using it.

[Gabriel Staples (ercaguy)]

Any news or progress on this? I've been fighting with installing ecryptfs now for days (first time using it) after installing Ubuntu 18 by following these instructions: https://www.howtogeek.com/116032/how-to-encrypt-your-home-folder-after-installing-ubuntu/amp/.

These instructions fail with rsync errors, but the usable encrypted home dir does get created, and when testing recovering an encrypted home from a live USB I ran into problem after problem (these instructions fail badly: https://www.howtogeek.com/116297/how-to-recover-an-encrypted-home-directory-on-ubuntu/amp/), but using the "Recovering Your Data Manually" instructions here (https://help.ubuntu.com/community/EncryptedPrivateDirectory#Recovering_Your_Data_Manually) does indeed work, so I feel somewhat better about using ecryptfs now.

My final solution may be to use the encrypted home I've got now, that is in fact recoverable from a live USB, but I need to go back and manually use rsync to copy from old home to new home again, fixing copy errors as I go which occur because some file names are too long since ecryptfs reduces max usable file name length since part of the name stores encryption info.

Am I on the right track to a good and viable solution? Or should I wait for Ubuntu 18 to fix this confusing mess and then reinstall using an official installer? Will fscrypt, which is supposed to be much faster and better than ecryptfs, ever be fully supported? When?

Is an encrypted home using ecryptfs known to be buggy or unusable in the long term even though it boots and appears to work now?

What should I do?

[Gabriel Staples (ercaguy)]

Also note I have a dual boot Windows/Ubuntu setup with UEFI and I don't want to break the dual boot.

[Gabriel Staples (ercaguy)]

Let me also link back to a few of my related comments (and related questions and answers) here, to kind of tie all of my efforts together:

1. https://askubuntu.com/questions/1029249/how-to-encrypt-home-on-ubuntu-18-04#comment1927239_1031509 and https://askubuntu.com/questions/1029249/how-to-encrypt-home-on-ubuntu-18-04#comment1927429_1031509
2. https://askubuntu.com/questions/659242/problem-with-ecryptfs-recover-private-mount2-failed/1088709#comment1926412_679565 and https://askubuntu.com/questions/659242/problem-with-ecryptfs-recover-private-mount2-failed/1088709#comment1926414_679565
3. https://askubuntu.com/questions/659242/problem-with-ecryptfs-recover-private-mount2-failed/1088709#comment1926414_679565

[Jarno Suni (jarnos)]

I have ecryptfs working in a computer that I upgraded to from 16.04 to 18.04. Boot is somewhat slow, but I do not know, if it is related.

[Tina Russell (tinarussell)]

I just installed Ubuntu 20.04 on a new machine and I’m a little frustrated that the installer made no mention of encrypting my home directory, which is what I was planning to do. I did a search and found out this is why. If ecryptfs is so broken and better alternatives exist, why does the installer not support any of them? Do I need to file a bug with the installer?

[Ahmad Gharbeia أحمد غربية (gharbeia)]

I've been using FScrypt on my Ubuntu 20.04 for a couple of months so far, and it's working fine.

[Gabriel Staples (ercaguy)]

Just a quick update on my end. This also addresses @Xavier Gnata's question here: https://bugs.launchpad.net/ubuntu/+source/ecryptfs-utils/+bug/1756840/comments/3

> Ok but what am I supposed to do on a dual boot machine??

I'm writing this all from memory. I did this a long time ago.

My solution is as follows on a Dual Boot machine.

Assume you have Windows on the machine.

1. Use Windows disk tools to make some space for Linux--ex: 120 GB of a 500 GB drive.
2. Fully encrypt Windows with Veracrypt. It is awesome and can do full partition encryption for your Windows partition, even while Windows is running! This might help: https://www.howtogeek.com/howto/6169/use-truecrypt-to-secure-your-data/.
2. Install Ubuntu 20.04 with LUKS volume encryption. You have to choose the "do something else" option when installing, and then you have to *manually* make an encrypted LUKS container. Inside that, you then install Ubuntu.

Done!

Now, when I boot, I get the Linux grub bootloader thing. If I choose Windows, I have to unlock the disk via a VeraCrypt unlock screen that pops up *before* Windows can even begin to boot. If I choose Ubuntu, it's similar: I have to decrypt the LUKS encryption on the screen that pops up *before* Ubuntu even begins to boot.

Works great. No more need for ecryptfs anymore in my particular case.

Note that on the Ubuntu option, I also make a big swapfile, as I describe here: https://askubuntu.com/questions/927854/how-do-i-increase-the-size-of-swapfile-without-removing-it-in-the-terminal/1177620#1177620.

This is in place of a swap *partition*. The swapfile has the advantage of being *within* the LUKS-encrypted volume, so it gets encrypted too.