Comment 144 for bug 509180

See also my comments #104, #128, #129, #131, #132, #134, #141, 143.

Ubuntu 11.10 Oneiric desktop new install, fully updated
shows file corruption when reading ecryptfs files.

Linux ubuntu 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

Reading the same read-only 18GB ecryptfs file system I've been using
above, I took md5sums of all the files and created a base file of 435,649
md5sum file entries.

I ran a script that looped for every file in the base file. It copied
each file to a tmp file, did an md5sum on the tmp file, and compared
that md5sum with the saved md5sum in the base file. I ran three looped
iterations of this script in parallel, each offset to start at a different
third of the md5sum base file. On the third iteration, all three scripts
found the same md5sum difference on the same file name:

MD5MISMATCH at line 62461 d4015d1962291b00abff6d14d8e71bb6 639f9b2e0f6bce8b287b879a1b0e17b5

All three scripts, running separately, came up with the same incorrect
md5sum for the tmp file copy, at three different times. The tmp copy was
the same size as the original. Comparing the tmp file with the original:

$ cmp -bl [original] [tmp]
 4865 0 ^@ 110 H
 4866 320 M-P 127 W
 4867 314 M-L 254 M-,
 4868 360 M-p 370 M-x
 4869 3 ^C 147 g
 4870 4 ^D 73 ;
 4871 360 M-p 237 M-^_
 4872 340 M-` 16 ^N
 4873 60 0 272 M-:
 4875 0 ^@ 347 M-g
 4876 0 ^@ 220 M-^P
 4877 0 ^@ 36 ^^
 4878 0 ^@ 347 M-g
 4879 240 M- 336 M-^
 4880 5 ^E 231 M-^Y
 4896 157 o 177 ^?

On the next five subsequent passes by each of the three scripts, no
md5sum changes were detected in any of the files, including the problem
file identified above. The corruption is intermittent.

From the kern.log file, here are some ecryptfs errors, but the errors
below do *not* correspond to the time that the script found the md5sum
difference (which was on 06:10am Oct 23):

Oct 20 14:57:49 ubuntu kernel: [ 65.128562] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
Oct 20 16:04:30 ubuntu kernel: [ 4066.054573] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 20 16:04:30 ubuntu kernel: [ 4066.054580] ecryptfs_readpage: Error decrypting page; rc = [-4]
Oct 20 16:47:30 ubuntu kernel: [ 6645.362178] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 20 16:47:30 ubuntu kernel: [ 6645.362263] ecryptfs_readpage: Error decrypting page; rc = [-4]
Oct 22 15:49:25 ubuntu kernel: [175960.571423] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 22 15:49:25 ubuntu kernel: [175960.571503] ecryptfs_readpage: Error decrypting page; rc = [-4]
Oct 22 16:12:46 ubuntu kernel: [177361.428710] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 22 16:12:46 ubuntu kernel: [177361.428735] ecryptfs_readpage: Error decrypting page; rc = [-4]
Oct 22 16:23:12 ubuntu kernel: [177987.700577] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 22 16:23:12 ubuntu kernel: [177987.700593] ecryptfs_readpage: Error decrypting page; rc = [-4]
Oct 22 16:31:57 ubuntu kernel: [178512.471610] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 22 16:31:57 ubuntu kernel: [178512.471613] ecryptfs_readpage: Error decrypting page; rc = [-4]
Oct 22 23:33:42 ubuntu kernel: [203817.735704] Valid eCryptfs headers not found in file header region or xattr region
Oct 22 23:33:42 ubuntu kernel: [203817.735717] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO
Oct 23 00:13:19 ubuntu kernel: [206194.602236] ecryptfs_decrypt_page: Error attempting to read lower page; rc = [-4]
Oct 23 00:13:19 ubuntu kernel: [206194.602243] ecryptfs_readpage: Error decrypting page; rc = [-4]