duplicate files/directories when mounted

[James P. Carter]

Binary package hint: ecryptfs-utils

I apologize if this is a duplicate. I did look through the other results. Encrypted home directory will not mount.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: ecryptfs-utils 83-0ubuntu3
ProcVersionSignature: Ubuntu 2.6.35-17.23-generic 2.6.35.2
Uname: Linux 2.6.35-17-generic i686
Architecture: i386
Date: Tue Aug 24 18:35:34 2010
EcryptfsInUse: Yes
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: ecryptfs-utils

[James P. Carter (jpcarter)]

Please let me know what else I may provide to you to troubleshoot this issue.

[Dustin Kirkland  (kirkland)]

Hi James,

Thanks for the report. Are you still having this issue?

This is an interesting one, I don't think I've ever seen anything like it.

Tyler, I have subscribed upstream, as I think you might be interested in this one... See James' screenshot. His ecryptfs mount has duplicated files/directories. Have you ever seen anything like this? Looks like potentially a bug in the encrypted filename name handling, maybe with multiple ciphertext mapping to a single cleartext?

[Joshua Lückers (joshualuckers)]

We'd like to figure out what's causing this bug for you, but we haven't heard back from you in a while. Could you please provide the requested information? Thanks!

[David Gross (dave-eorbit)]

I'm having what seems to be a similar problem.

I recently upgraded to Ubuntu 11 and a few days later my ext4 filesystem began misbehaving in a weird way.

In my home directory, many subdirectory names are duplicated verbatim. Here's an ls -F excerpt:
...
Desktop/
Desktop/
Documents/
Documents/
Downloads/
Downloads/
...
I can no longer access files in those directories (if I ls the directory, it appears empty; I can cd to it, but there's nothing inside).

Not all of the directories are duplicated/damaged like this, but most are. A few non-directory files are also duplicated in this fashion, so for example:
-rw-r--r--1 dgross dgross 95 2011-05-08 16:16 .dmrc
-rw-r--r--1 dgross dgross 95 2011-05-08 16:16 .dmrc
But I can read the contents of those files.

Because of the empty/damaged directories I can only start up in crippled recovery mode (too many essential files are impossible to find). Also many important content files are missing and I'd love to be able to recover them.

-----
> sudo fsck.ext4 /dev/sda1 -f -v
e2fsck 1.41.14 (22-Dec-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

250236 inodes used (0.84%)
5514 non-contiguous files (2.2%)
559 non-contiguous directories (0.2%)
# of inodes with ind/dind/tind blocks: 0/0/0
Extent depth histogram: 216180/243
23781951 blocks used (19.98%)
0 bad blocks
1 large file
181532 regular files
26200 directories
59 character device files
26 block device files
0 fifos
423 links
42407 symbolic links (33715 fast symbolic links)
3 sockets
--------
250650 files

[David Gross (dave-eorbit)]

This looks like a possibly useful datapoint that showed up in dmesg at some point during my ongoing debugging / diagnosis attempts:

> dmesg | grep ryptfs
[ 782.510299] Mount on filesystem of type eCryptfs explicitly disallowed due to known incompatibilities

[Tyler Hicks (tyhicks)]

David - That is an error condition that would prevent a 2nd eCryptfs mount from happening. I don't think that is the cause of the problem, since the 2nd mount attempt simply failed.

Have either of you manually mounted your encrypted home directory using the `mount -t ecryptfs /home/.ecryptfs/$user/.Private /home/$user/` method? I'm guessing that at some point a different filename encryption key was used to do a mount and now you're stuck with the duplicated directories (Templates, Videos, Ubuntu One, etc.) that are created by some magical component (to me, anyways) the first time you log in.

[David Gross (dave-eorbit)]

For me this took place after an ordinary login without manually mounting the directory. Since then I've used ecryptfs-recover-private to mount it privately so I could copy out as much data as I could to some other directory (it, too, sees the corrupted filesystem with duplicate directories and the like).

I just went into ...home/.ecryptfs/myname and did an ls -laF .Private/ and was surprised to see both encrypted and unencrypted names of the duplicated files and directories!

e.g.
...
drwxr-xr-x 2 1000 1000 4096 2011-05-07 05:12 Downloads/
drwxr-xr-x 22 1000 1000 4096 2011-05-05 03:11 ECRYPTFS_FNEK_ENCRYPTED.Gobbledygook--/
...

I'm not familiar enough with ecryptfs to know for sure, but that doesn't smell right to me.

[David Gross (dave-eorbit)]

I'm tempted to move all of the unencrypted directories/files out of this directory to some other place and then trying to remount the encryptfs directory to see if I can then see my old stuff again.

[David Gross (dave-eorbit)]

That seems to have done the trick. Things are still acting slightly awkwardly, but it looks like all of my data is still there, thank goodness. Hopefully this trail of breadcrumbs will help other folks...

[Launchpad Janitor (janitor)]

[Expired for ecryptfs-utils (Ubuntu) because there has been no activity for 60 days.]

[Attila Lendvai (attila-lendvai)]

i've also just experienced it with ubuntu 11.04

my usecase: i've set up separate users for skype and chrome, and use two such scripts to start them:

#!/bin/bash

xhost +local:
gksu --su-mode --login --user chrome "export SDL_AUDIODRIVER=alsa; google-chrome"

i assume it's reproducible, because both 'skype' and 'chrome' users have duplicate filenames, and here's an excerpt from the ls -l:

# ll /home/.ecryptfs/chrome/.Private/
[...]
-rw------- 1 chrome chrome 12288 2011-09-08 20:47 ECRYPTFS_FNEK_ENCRYPTED.FXbTsLaTpT.vrESWNmnq7nO2qAl3QvNgaCzOfcgaNrfvuZAFqgbyt9TOW6FxK3GtWwQL01lCyOyY7V2-
-rw------- 1 chrome chrome 12288 2011-09-08 22:47 ECRYPTFS_FNEK_ENCRYPTED.FXbTsLaTpT.vrESWNmnq7nO2qAl3QvNgaCzOr-KAoNABxcFpoqEAOROpm3ucwy8d04B.NOSPhJt-2GY-
-rw-r--r-- 1 chrome chrome 12288 2011-09-08 14:01 ECRYPTFS_FNEK_ENCRYPTED.FXbTsLaTpT.vrESWNmnq7nO2qAl3QvNgaCzOr-KAoNABxcFpoqEAOROpm9g1LBboIxh9i6c9kXOSk16-
-rw------- 1 chrome chrome 12288 2011-09-08 20:44 .esd_auth
drwx------ 5 chrome chrome 4096 2011-09-08 20:44 .gnome2/
-rw-r--r-- 1 chrome chrome 12288 2011-09-08 22:46 .gtk-bookmarks
drwx------ 2 chrome chrome 4096 2011-09-08 20:44 .gvfs/
[...]

as far as i understand i shouldn't see unencrypted stuff there, right?

[Attila Lendvai (attila-lendvai)]

i was sloppy, sorry. my skype user was misbehaving, but i don't see the corruption there, so it might have been something else.

but look at this, it's in the broken one (chrome):

chrome@ed101:~$ rm -rf .config/
chrome@ed101:~$ ll -d .config
drwx------ 7 chrome chrome 4096 2011-09-08 20:45 .config/
chrome@ed101:~$

doesn't matter how many times i try to delete that dir, it remains and no error is printed.

another inconsistency (look at the size difference!):

chrome@ed101:~$ du -s /home/chrome/
7100 /home/chrome/
chrome@ed101:~$ du -s /home/.ecryptfs/chrome/.Private/
994572 /home/.ecryptfs/chrome/.Private/
chrome@ed101:~$ mount
[...]
/home/chrome/.Private on /home/chrome type ecryptfs (ecryptfs_check_dev_ruid,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_unlink_sigs,ecryptfs_sig=c266af0d5aa7fbc7,ecryptfs_fnek_sig=dfe1799fd5f07bdd)
chrome@ed101:~$

then i logged out, then back in, and .config contains the expected files (not sure if they are intact, though):

chrome@ed101:~$ du -s /home/chrome/.config
986648 /home/chrome/.config

hth.

[Rich Wales (richw)]

I got bitten by this bug tonight. Like Dave Gross, I discovered that my .Private directory had a bunch of empty directories with unencrypted names in it -- corresponding to the directory names that were duplicated (and empty) when my eCryptfs directory was mounted. I fixed the problem by renaming my .Private directory (with the encrypted directory unmounted, of course!), creating a new .Private directory (with identical ownership and permissions), and moving all (and only) the ECRYPTFS* files/directories from my old .Private directory to the new directory. After doing this and remounting my encrypted directory, everything was fine. How the problem happened in the first place, I have no idea.

[Wacław Jacek (waclawjacek)]

This has happened to me recently. I have logged in and all my GNOME preferences went missing, and my home directory contained a lot of directories similar to the ones in comment #13. After re-logging in (maybe a few times, I might have not checked my home dir), my GNOME settings were still missing, but my directories and files are back. It seems to me as if after one login my home directory wasn't mounted properly and GNOME has recreated the settings and default directories, as they seemed to be missing, and when it's finally mounted them properly, it uses the ones it's created, since it now has duplicate directories (maybe it's overwritten the old ones, I can't tell because regardless which of the duplicate directories I enter, I end up in the same directory).

[Dustin Kirkland  (kirkland)]

Hmm, this bug is still bothering me. I haven't seen it or reproduced it. For those affected, could you please note your version of ecryptfs-utils, linux, and Ubuntu? Thanks.

[Dustin Kirkland  (kirkland)]

Okay, morning this bug a duplicate of bug #823775, where lightdm was doing some work (creating directories, files, etc) in the user's home directory as root *before* the user's homedir was mounted, I think.

If you can still reproduce this, please let us know and we'll de-dupe it. Thanks!

[tolakvoa (tolakvoa)]

I too have a problem similar to David's but I still don't have a clue how to fix it because there are so many answers here and i did tried to follow some of the solutions but no success. My home folder is in second hardrive and I encrypted during installation but now when I use ls -F I see duplicated folders but I can't get to my old files. Which the exact method should I use to get my files back. Can you please how could I get my files back.

[Rich Wales (richw)]

Examine your directory WITHOUT mounting the encrypted partition. That is, look at the .Private directory which contains a bunch of "ECRYPTFS..." files and directories. In your case, you'll see a mix of "ECRYPTFS..." files/directories (containing your real data), and other directories with regular-looking names but no content.

If you mount the encrypted partition, then whenever there is an encrypted item whose name decrypts to something non-encrypted in the same directory, the system will choose the non-encrypted (and useless) item instead of the real (encrypted) item.

You need to VERY CAREFULLY get rid of everything in your .Private directory that does NOT have an "ECRYPTFS..." name.

You can either remove the offending junk directories by hand (if you're VERY confident and VERY careful) -- or you can do what I did. I renamed the .Private directory, created a new .Private directory (with identical ownership and permissions), and then went into the original (renamed) directory and moved everything with a name starting with "ECRYPTFS" into the new .Private directory.

After doing this, you should be able to mount the encrypted partition, and everything should hopefully be OK.

[tolakvoa (tolakvoa)]

Rich,
I will try you suggested and I will responsible on my own action and I will let you know the result. I have to re read what you said because basically i was just done fast reading your comment and I still don't understand much but I will try.
Thanks

[tolakvoa (tolakvoa)]

Rich,
I did what you suggested by rename .Private then recreate .Private folder again and during moving back all rename .Private folder I got error message "file name too long" but I just let it go and I saw my hd light was working hard so I did not interrupt it. When it is done no other error message then I immediately reboot and voala I got my files back. Thank You and Thank You.

[Nils Durner (ndurner)]

Happened out of the blue (don't remember if I did an online update yesterday) to me on Oneiric today: logged in, had an empty desktop & saw lots of ECRYPTFS_ objects in my Personal Folder. Rebooted, all files in the standard folders seemed to be gone. At closer inspection, I noticed that all those folders were duplicated.

#20 fixed it, thanks!

Keywords: empty folders; empty profile; deleted files in Documents, Downloads

[Nicolas (nipolas)]

#20 fixed it too

this bug came out of nowhere
using ubuntu 11.10

[Roland (roland-breedveld)]

Same issue with 12.04
#20 solves the problem

[Carl Menke (halfburnttoast)]

This just happened to me tonight under xubuntu 12.04 x64. It did exactly the same thing that #23 said. I wasn't really doing anything before it happened. No updating or installing. Just rebooted and it happened.

Following #20 solved the problem.

[Talion (talion)]

Same issue here on Xubuntu 12.04, out of the blue after a normal login. (Empty desktop, duplicate files in ~/home, default desktop settings restored, all personal launchers and keyboard shortcuts disappeared, ...)
#20 solved the problem (I just deleted everything that had no ECRYPTFS in its name, try "find /home/.ecryptfs/USERNAME/.Private/ | grep -v ECRYPTFS" for a list of such files)
During the last weeks I experienced daily to hourly crashes/freezes of xfce4-panel. Maybe this was already caused by non-available files due to the damaged encryption - I'll see whether it continues or not.

cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.1 LTS"

dpkg -l | grep ecrypt
ii ecryptfs-utils 96-0ubuntu3

uname -a
Linux tl 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:04:05 UTC 2012 i686 i686 i386 GNU/Linux

[Andrey (andreykostromin)]

Thanks, Talion. You saved my time!!!!

andrey@DARK:~$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.10
DISTRIB_CODENAME=quantal
DISTRIB_DESCRIPTION="Ubuntu 12.10"
NAME="Ubuntu"
VERSION="12.10, Quantal Quetzal"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu quantal (12.10)"
VERSION_ID="12.10"
andrey@DARK:~$ dpkg -l | grep ecrypt
ii ecryptfs-utils 100-0ubuntu1 amd64 ecryptfs cryptographic filesystem (utilities)
ii libecryptfs0 100-0ubuntu1 amd64 ecryptfs cryptographic filesystem (library)
andrey@DARK:~$ uname -a
Linux DARK 3.5.0-21-generic #32-Ubuntu SMP Tue Dec 11 18:51:59 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

[TiguSoft (opensource-m3)]

Same issue is present on Ubuntu 12.04.01.
We are now repairing this file system, fell free to email us if you want to debug the problem.

[TiguSoft (opensource-m3)]

many people confirmed. Our customer also had same bug, and we fixed it like in comment nr 20 - by (while unmounted eCryptfs, go into that .Private eg /home/your_user_name/.ecryptfs/.Private and there move aside eg to /home/your_user_name/BROKEN/ all the directories and files that do NOT match the usual format ECRYPTFS... , then mount again or restart)

[TiguSoft (opensource-m3)]

btw imo this bug is Grave, it wrecks users computers, they think their data might be gone, and need to seek IT specialists to fix it (unless they are enough linux guys to find this solution and perform it carefully)