files in eCryptFS Private directory get corrupted

[Christian Mertes]

I'm using a .Private directory to encrypt parts of my home directory. I'm using ext4 in my home and in my ~/Private. Some files turned became corrupted for no apparent reason in that trying to read them produced an "Input/output error" on the console and this in the syslog:

Mar 17 12:08:55 Saffron kernel: [ 2427.492737] Valid eCryptfs headers not found in file header region or xattr region, inode 306879
Mar 17 12:08:55 Saffron kernel: [ 2427.492757] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

This includes some chromium files (I'll probably survive that) and some files from my PhD thesis (not funny if it happens before I can git push that stuff out of reach for ecryptfs to mess with).

I found this post from 2009 (!) that matter-of-factly asserts that ecryptfs volumes corrupt files when the disk runs out of space: https://answers.launchpad.net/ubuntu/+source/ecryptfs-utils/+question/60020

I did indeed once run out of disk space but there are younger corruptions.

I also failed to figure out how to fsck an ecryptfs volume. I only found people asking that question with zero answers.

At the current state of affairs I'm seriously considering to switch to full disk encryption and to tell everyone to stay as far away from ecryptfs as possible. If necessary, run.

But maybe I'm missing something regarding the missing fsck and that file corruption bug is easily fixed.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: ecryptfs-utils 96-0ubuntu2
ProcVersionSignature: Ubuntu 3.2.0-18.29-generic 3.2.9
Uname: Linux 3.2.0-18-generic x86_64
ApportVersion: 1.94.1-0ubuntu2
Architecture: amd64
Date: Sat Mar 17 12:10:08 2012
EcryptfsInUse: Yes
InstallationMedia: Lubuntu 12.04 "Precise Pangolin" - Alpha amd64 (20120201.1)
ProcEnviron:
 LANGUAGE=en_GB:en
 TERM=xterm
 PATH=(custom, user)
 LANG=en_GB.UTF-8
 SHELL=/usr/bin/fizsh
SourcePackage: ecryptfs-utils
UpgradeStatus: No upgrade log present (probably fresh install)

[Christian Mertes (cmertes)]

Oh I should mention that I checked the volumes that let me do that and also ran a smartctl --long check on the disk. Everything seems fine other than on that black box ecryptfs volume.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Mertes (cmertes)]

It's not a duplicate of bug #509180. #509180 is about some trailing garbage appearing at the end of files, then you remount and everything is shiny again. This bug is about files getting broken and staying broken and all the data in them is lost! Please remove the duplicate status and set the importance on critical instead. Not that I use this encryption layer anymore but if that's not a critical bug in the context of the ecryptfs-utils package that deserves more attention then what is?

[Charl P. Botha (cpbotha)]

I'm copying a part of my comment on bug #509180, because in fact what I'm seeing on my systems is what's described in this bug report here.

In short, I'm also seeing the "0-length lower file on ext4 leads to IO Error in mounted ecryptfs" bug. It's truly shocking that Ubuntu 12.04 LTS is shipping with this pathological behaviour. At least 247 of my files are at risk. Many users could be losing data and not even know about it.

I don't dare delete the lower files, because I'm not sure what's going to happen to the upper files, and then Dropbox might sync that to all my other copies. I've currently stopped using the two laptops in question.

Either a SRU needs to be done ASAP, or the option during installation to encrypt one's home directory needs to be disabled (possibly only when using ext4).

More information on one of my laptops:

$ uname -a
Linux cpbotha-e6410 3.2.0-24-generic #38-Ubuntu SMP Tue May 1 16:18:50 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

$ find $HOME/.Private/ -size 0c -exec ls '{}' \; | wc -l
253

Indeed, my dropbox is stuck on indexing 247 files (this is how I noticed). When I go to one of the files in question:

$ ls -la envedit.exe.manifest
-rw-rw-r-- 1 cpbotha cpbotha 0 May 14 17:17 envedit.exe.manifest

$ cat envedit.exe.manifest
cat: envedit.exe.manifest: Input/output error

... and then in my dmesg:

[ 6554.071505] Valid eCryptfs headers not found in file header region or xattr region, inode 4882470
[ 6554.071511] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

In other words, I have at least 253 files (some of them more important than envedit.exe.manifest) that are at risk.

[Kai Falkenberg (wooosh)]

Hi,

I can confirm this:

$ uname -a
Linux ninux 3.0.0-19-generic #33-Ubuntu SMP Thu Apr 19 19:05:57 UTC 2012 i686 i686 i386 GNU/Linux

$ find $HOME/.Private/ -size 0c -exec ls '{}' \; | wc -l
4

[ 444.916666] Valid eCryptfs headers not found in file header region or xattr region, inode 2890741
[ 444.916670] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

and
$ cat $HOME/.ICEauthority
cat: .ICEauthority: Input/Output error

Can't login to Xfce with this user.. :(

[Laurent Dinclaux (dreadlox)]

$ uname -a
Linux lion 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

$ find $HOME/.Private/ -size 0c -exec ls '{}' \; | wc -l
165

[Juan Moyano (wincus)]

Same thing here:

Linux inspiron 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

jon@inspiron:~$ dpkg -l | egrep ecrypt
ii ecryptfs-utils 96-0ubuntu3 ecryptfs cryptographic filesystem (utilities)
ii libecryptfs0 96-0ubuntu3 ecryptfs cryptographic filesystem (library)

jon@inspiron:~$ shred .gconf/apps/gnome-terminal/profiles/Default%gconf.xml.new
shred: .gconf/apps/gnome-terminal/profiles/Default/%gconf.xml.new: fallo al abrir para escritura: Error de entrada/salida

I did run out of space a couple of weeks ago. I tracked down the files by their inode number and found at least 20 files. I had to delete the files manually. There was real damage :(

I'm wondering if I should step out encrypted partitions

[Félim Whiteley (felimwhiteley)]

This is also effecting me but I _DID_NOT_ run out of space, in fact I've about 500GB free. System is upgrade from 11.10 to 12.04 Kubuntu.

kernel: 3.2.0-26-generic #41-Ubuntu SMP Thu Jun 14 17:49:24 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

ecryptfs-utils: 96-0ubuntu3

Same characteristics, .Private with 0 size files, so far some firefox files that didn't casue a problem and a coupld of cache files but I've notied a few times Akonadi (Kubuntu 12.04) through up strange errors which may be related.

A recurring one is ~/.mozilla/firefox/m1cop2xb.default/urlclassifier3.sqlite-journal
&
.mozilla/firefox/m1cop2xb.default/permissions.sqlite-journal

[Juan Rial (OpenERP) (jri-openerp)]

Same problem here. Mostly with the Skype config files, but today I lost a couple of LibreOffice 3 config files which LO3 raises error dialogs over if it can't find or read them.

So I doubt those files would've been empty, especially considering that LO3 ships with populated versions of those files in /usr/lib/libreoffice/basis3.4/share/. If I'm correct, it also means this bug is not a duplicate of #911507: that one concerns empty files, while this one's about real data loss.

[Iwan Sapoetra (isapoetra)]

dmesg full with message
Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

uname -a
Linux cgii 3.5.0-8-generic #8-Ubuntu SMP Sat Aug 4 04:43:06 UTC 2012 i686 i686 i686 GNU/Linux

find $HOME/.Private/ -size 0c -exec ls '{}' \; | wc -l
find: `/home/[username]/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWYJm-6tEn3wF-RFi3YfsmzwbVccigwCXElHEUNGfr0EuFuG3wXSMTlb-U--': Permission denied

so i use command
sudo chown -R ~ [username]
/home/[username]/.cache/dconf/user Permission denied

I remove /home/[username]/.cache/dconf/user with rm -rf ~/.cache/dconf/user

and the message on dmesg gone...

[jbwiv (bugs-sourceillustrated)]

So with the find...exec...wc command string above, I find 189 files in my directory. How can I track these encrypted files down the their unencrypted counterparts in order to tell what files are affected?

[Alan Boudreault (aboudreault)]

I'm also affected with this bug. This is very annoying since that I'm getting unexpected segfault in libssl and libcryto.

[karla (karla-9)]

I'm also affected by this bug.

I upgraded from 11.10 to Ubuntu 12.04 a week ago. My disk did *not* run full.

I stumpled upon these syslog messages:
Valid eCryptfs headers not found in file header region or xattr region, inode 133759
Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

I had a look into the syslog file because firefox was not able to show my bookmarks anymore and Thunderbird started to download my entire mailbox again. Till now I was not able to restore my bookmarks.

The log lines above show up especially when I open firefox. Firefox config/profile files seem to be affected (places.sqlite).

Is there anything one can do to fix this or do I have to do a complete reinstall?

[Christian Mertes (cmertes)]

@jbwiv What unencrypted counterparts?

@karla The syslog messages are normal for this bug, see the original description. What you can do is resize your partition with gparted, create a new crypto partition using LUKS, copy all data to the new crypto partition and reconfigure pam_mount to use crypt instead of ecryptfs. There are many howtos on the Net, just one random one: http://blog.adamsbros.org/2009/06/16/quick-guide-to-luks-encrypted-home-volumes/

[Florian Rathgeber (florian-rathgeber)]

@jbwiv no need to search in $HOME/.Private, just search ~ and you'll get the actual files.

[Beowulf (s-highlander)]

I confirm this bug.

[Alan Boudreault (aboudreault)]

I'm about to setup a new laptop with Kubuntu 12.04.1. I'm afraid to reenable the home encryption. Is this bug going to happen again? What was its cause? Should I just use the luks setup?

[Christian Mertes (cmertes)]

You should definitely use LUKS instead of ecryptfs, Alan. There is no indication that this bug got fixed or even looked into since it has been reported. If by "cause" you mean what triggers this bug: when the FS containing the ecryptfs files runs out of space, files in the ecryptfs volume get corrupted. If you don't use any unencrypted area on your home you might be safe but then there is no reason whatsoever to use ecryptfs in the first place.

[Tyler Hicks (tyhicks)]

This has been tracked in various other bugs and is fixed in recent kernels. Sorry for the confusion of this bug not being marked as a dupe of one of the others. There were a few specific bugs that culminated into the broader problem reported in this bug.

The way eCryptfs handled full disk situations and opening lower files that didn't contain the proper eCryptfs metadata has changed over the course of time but here are the relevant patches that have went upstream to address the issues mentioned in this bug report:

821f749 eCryptfs: Revert to a writethrough cache model
e3ccaa9 eCryptfs: Initialize empty lower files when opening them
8bc2d3c eCryptfs: Unlink lower inode when ecryptfs_create() fails

For Ubuntu users, these fixes have all been picked up in the 12.04 LTS kernels.

[Alan Boudreault (aboudreault)]

Thanks you for your answer, I'm glad this has been fixed!

[Tyler Hicks (tyhicks)]

For completeness, I wanted to point out two more patches which should be added to the list I gave in comment 20. 821f749 introduced a regression and these two patches are needed to fix the regression:

64e6651 eCryptfs: Call lower ->flush() from ecryptfs_flush()
7149f25 eCryptfs: Write out all dirty pages just before releasing the lower file

Actually, 7149f25 is the only one required but I recommend 64e6651, as well.

[Larry Tate (cathect)]

Using eCryptfs-utils (111-0ubuntu7) from the repos.

I have an encrypted directory containing three types of files: .png, mp3, .md. Upon mounting, the .png and .mp3 files are corrupted and are not viewable/playable. The .md text file, however, works as expected.

[Attila Lendvai (attila-lendvai)]

.md files are text files, right? the corruption there may be non-obvious (e.g. a couple of characters), while the programs processing mp3 or png files can go awry even when one bit gets corrupted.