eCryptfs should initialize existing empty files at open()

[Tyler Hicks]

Empty files left in the lower filesystem cause eCryptfs to return error codes and log error messages when eCryptfs tries to read the metadata. This has been the mode of operation since the beginning of eCryptfs. There is no eCryptfs metadata available, so it seemed appropriate to refuse to open the file.

However, more and more users are having trouble with this poor design. It would not be too difficult to convert the empty file into a proper eCryptfs file in the open() path, just as is done in the create() path.

Currently, users see these messages logged to their syslog:

Valid eCryptfs headers not found in file header region or xattr region
Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

Transparently converting the empty file to an eCryptfs file and successfully completing the open() request will greatly improve usability.

WORKAROUND: see comment #19 https://bugs.launchpad.net/ecryptfs/+bug/911507/comments/19

[Mikko Rantalainen (mira)]

I don't agree with the reasoning here. If the user has disabled passthrough mode, then any file in the lower filesystem that is not regognized by eCryptfs is an error.

eCryptfs should have an fsck which could remove such empty files from lower filesystem or convert those files to encrypted empty files. However, I don't think that filesystem should automatically do such a tricks. (Compare to existing filesystems such as ext4 - it does not automatically try to run fsck and fix logical problems in the filesystem.)

I'd prefer a single error message to syslog about the problem and then automatically remounting ecryptfs read-only. Then I would have to run fsck to fix the issue. Hopefully the syslog entry would point me towards that target.

[Adam Porter (alphapapa)]

Considering the frequency of this bug and the ease of causing it, I think it would be very bad to automatically remount read-only. Running apps would suddenly stop working in unpredictable ways and the user would be left bewildered, and even unable to save his data before logging out to solve the problem.

What would the purpose of a 0-byte file be? Surely it would be the result of a bug. Converting it to a proper file would restore correct functionality, but dying would cause frustration, inconvenience, and, potentially, data loss. Isn't the Right Thing to do to fix the problem and keep working? Just output a warning to syslog and continue on.

[Tyler Hicks (tyhicks)]

On 2012-01-04 08:30:57, Mikko Rantalainen wrote:
> eCryptfs should have an fsck which could remove such empty files from
> lower filesystem or convert those files to encrypted empty files.
> However, I don't think that filesystem should automatically do such a
> tricks. (Compare to existing filesystems such as ext4 - it does not
> automatically try to run fsck and fix logical problems in the
> filesystem.)

Thanks for your thoughts/feedback on this issue.

Keep in mind that we already do this in the ->create() function. This
would be an extension of that where if we didn't see the metadata in our
->open() function and the lower inode size is 0, we'd generate new
metadata and write it out.

> I'd prefer a single error message to syslog about the problem and then
> automatically remounting ecryptfs read-only. Then I would have to run
> fsck to fix the issue. Hopefully the syslog entry would point me towards
> that target.

I'd prefer no error messages to syslog (filesystems really shouldn't be
so chatty) and for things to be handled transparently for the user.

Remounting read-only is definitely not an option. Too many users are
starting to hit this issue and remounting read-only would be a usability
nightmare. It makes sense for power users and admins, but non-advanced
users would have a lot of trouble figuring out how to mediate the issue.

What if this functionality was only enabled when a mount option is
specified? By default, eCryptfs would continue with its stubborn ways of
erroring out on empty lower files, but a mount option would be available
to allow them to be converted to proper eCryptfs format in ->open(). Any
thoughts?

[Mikko Rantalainen (mira)]

The reason I would still suggest mounting read-only on any error is to prevent silent data loss (e.g. some problems in eCryptfs were found because people put their git repositories on eCryptfs and git detected error in the fs/storage. I'd hate to have silent data loss by design only because end user may end up failing to login with corrupted file system). I'd be somewhat okay with non-default mount option that fixes the issue by e.g. removing any files that are not encrypted or just empty ones.

Note that ecryptfs always has a bug when there's an invalid file in the lower filesystem (or the user has manually messed with the lower filesystem and it that case he should expect mounting to fail). If the eCryptfs is still too unstable to default to mounting read-only on any error, then it really should not used as user home mount. If the only issue is caused by previous (buggy) versions of ecryptfs leaving random corrupted files in the lower filesystem, then system (Ubuntu) should run one-time script to fix those issues (namely, removing empty files in lower filesystem) before trying to mount the user's home with eCryptfs.

[Hasse Bylov (hassebylov)]

Very frequently my /var/log/syslog gets spammed with content similar to this:
...
Mar 13 09:49:14 lab kernel: [ 3859.532111] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO
Mar 13 09:49:14 lab kernel: [ 3859.532133] Valid eCryptfs headers not found in file header region or xattr region, inode 857952
Mar 13 09:49:14 lab kernel: [ 3859.532140] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO
Mar 13 09:49:28 lab kernel: [ 3874.017290] Valid eCryptfs headers not found in file header region or xattr region, inode 857954
Mar 13 09:49:28 lab kernel: [ 3874.017301] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO
Mar 13 09:49:28 lab kernel: [ 3874.017331] Valid eCryptfs headers not found in file header region or xattr region, inode 857954
Mar 13 09:49:28 lab kernel: [ 3874.017338] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO
...

On other machines I've been unable to login via another TTY terminal (<ctrl> <alt> <f1>), because of this ongoing spamming from the kernal to STDOUT. I'm actually considering getting rid of eCrypt FS on all my machines because of this.

Any ideas on how to solve it?

[Vadim Peretokin (vperetokin)]

I've enabled home encryption when installing Ubuntu 11.10, and have not modified any encryption settings after whatsoever.

Yet my logs files are spammed immensely with these messages. Is this not an issue that the default settings are spamming out so much?

[Rocko (rockorequin)]

Ditto with the spam. I'm getting the message every two seconds at the moment, which is ridiculous.

[David Moreno Fernández (linuxonrails)]

I have the same problem:

Apr 13 21:19:10 euclides kernel: [46490.571078] Valid eCryptfs headers not found in file header region or xattr region, inode 8952199
Apr 13 21:19:10 euclides kernel: [46490.571081] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

[Tyler Hicks (tyhicks)]

This has obviously become a usability issue.

I respect your opinion on not doing this kernel side, Mikko, but this is something that can and should be handled transparently so that the user don't have to mess with it.

I'm going to look into doing the conversion from a zero length lower file to a proper lower file (with eCryptfs metadata).

[Mikko Rantalainen (mira)]

Tyler Hicks: I respect your decision - you are the maintainer of the package, after all. My opinions are only my opinions.

The point I was trying to make is that is not acceptable to paint over the problem of invalid files popping up in the lower file system. Yes, it causes a big usability problem but the real cause for the issue should be identified instead of hiding the issue. Please, at least spam the syslog for each file you automatically "fix" so possible data loss is not done silently. Reporting the fixes is a bit problematic, though, because you don't want to leak the unencrypted filename to the log and the targeted end user cannot deal with anything more cryptic, I'm afraid.

Would it be possible to have encrypted "lost+found" directory which contained original copy of every "fixed" file?

[Tyler Hicks (tyhicks)]

On 2012-04-16 06:44:17, Mikko Rantalainen wrote:
> The point I was trying to make is that is not acceptable to paint over
> the problem of invalid files popping up in the lower file system. Yes,
> it causes a big usability problem but the real cause for the issue
> should be identified instead of hiding the issue.

Fair point. However, there are scenarios when the real cause is not
fixable.

For example, there is a small window of time in ecryptfs_create(),
between the lower inode being created and eCryptfs metadata being
generated and written out to the lower file. A loss of power between
those two events will likely result in a zero length lower file.

> Please, at least spam the syslog for each file you automatically "fix"
> so possible data loss is not done silently.

I don't see how data loss is possible when fixing up zero length files.
The lower file is empty, after all.

Whatever lead to the zero length lower file could theoretically be a
source of data loss, but that would be a bug that would be fixed.

I also don't think end users care to know that an empty file was fixed.
There is no action for them to take and the error message will only fill
up the logs and possibly confuse them. A debug level message may be
appropriate, though.

> Reporting the fixes is a bit problematic, though, because you don't
> want to leak the unencrypted filename to the log and the targeted end
> user cannot deal with anything more cryptic, I'm afraid.

If needed, this can be handled by printing the inode number.

>
> Would it be possible to have encrypted "lost+found" directory which
> contained original copy of every "fixed" file?

I don't see the point since they would all be empty files.

[DanielW (danielw)]

Ubuntu 12.04 LTS.

I've also the same problem:
May 11 09:36:34 WOL kernel: [ 1061.171752] Valid eCryptfs headers not found in file header region or xattr region, inode 11378750
May 11 09:36:34 WOL kernel: [ 1061.171755] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

....with lots of entries in the logfile!

[Ivaylo Hubanov (sniffski)]

Hi guys,
I got the same problem...
If you want to find the file which is causing this issue just type $ find -inum {INODE} this will find the file that is with zero length and you can decide if you can delete it or no.
My issue was with .Skype/shared.xml
For some reason this file changed it's attributes and was not accessible. I just removed it and my problem was solved.
Hope this info help anyone who have the same issue.

[reliable-robin-22 (nicolasdiogo)]

same error using Ubuntu 12.04 amd64

error message on dmesg:

[40611.456052] Valid eCryptfs headers not found in file header region or xattr region, inode 3936613
[40611.456056] Either the lower file is not in a valid eCryptfs format, or the key could not be retrieved. Plaintext passthrough mode is not enabled; returning -EIO

attachment has info about the packages that i have installed

it sounds like something that *really* needs fixing.

please let me know if i you require further info

thanks,

Nicolas

[reliable-robin-22 (nicolasdiogo)]

also the file in question was the same:

.Skype/shared.xml

[Kai Webber (kai11)]

I got those messages after I run out of disk space, so it's another possible use case for Tyler.

[Tyler Hicks (tyhicks)]

A fix was sent out for review:

http://thread.gmane.org/gmane.comp.file-systems.ecryptfs.general/231

[Patrik Båt (failure)]

rm .Skype/shared.xml did the trick

[Ivo Wever (ivo-wever)]

Here's a way to find the offending file:

find ~ -max-depth $N -inum $INUM

where $INUM is the inode number reported in the error message and $N is the depth you want to search. I started searching without depth, but nothing was found after searching for several minutes. When I searched with depth 2, then 3, two Skype config files turned up immediately. Later some chromium config file was the wrongdoer. Replace ~ by / if everything is encrypted and not just your home directory. Adjust as needed.

[Félim Whiteley (felimwhiteley)]

I keep getting these showing up, as I delete each ona new one appears. Is there any update on a fix for this?

Seems more folk are still getting it, I marked them as duplicates and linked them here. It's the same issue. I've not run out fo space, some people have, but it's mainly temp files from Skype, Firefox etc.

[Ivo Wever (ivo-wever)]

There's a typo in my previous comment: the working way to find the offending file is:

find ~ -maxdepth $N -inum $INUM

Concerning the Skype files: removing them doesn't work, as they keep being recreated. Instead of removing them, I've now added an empty XML comment <!-- --> to each, which works just fine.

[mr.goose (editor-garfnet)]

Many thanks to those who suggested "find ~ -maxdepth $N -inum $INUM" method.

In my case it found:-
~/Documents/mozilla/firefox/garf/webappsstore.sqlite-journal
(My emails, browser settings and bookmarks are very important to me. Therefore I keep my Thunderbird and Firefox stuff where I can see it, so it gets backed up, and so that it doesn't get lost if I have to move in/out of a PC very quickly.)

However I did not set a maxdepth. I also needed to sudo it because I regularly backup my sites and several other remote servers, via rsync over ssh, to a subdirectory of my home directory. I retain the permissions of the backed up files because it can be a pain figuring out who owns what, should I need to restore anything in a hurry. (Yeah I know I could/should tarball it, but speed is of the essence here!)

Anyway, the inode I was seeking was 21933851. So the command I used was:-
sudo find ~ -inum 21933851

This worked a treat. Without the sudo - I got lots of "permission denied"errors from my backup files.

HTH someone. Best wishes, G.

[Shevek (r-launchpad-anarres-org)]

.skype/shared.xml for me too

[Christian Mertes (cmertes)]

Félim Whiteley, did you read the other bugs before marking them as duplicates? Bug #957843 is about data loss (and none of the devs seems to care -- man am I happy I left ecryptfs for good), whereas in this bug people are discussing whether this or that fix might or might not result in data loss. To me this does not seem to be the same issue. If it is: boy are you relaxed round here.

Summary, in case somebody missed this -- I'd personally call it rather important -- issue for a file system: ECRYPTFS IS LOSING DATA!

[Tyler Hicks (tyhicks)]

This fix was merged into Linus' tree. It should be released in 3.6-rc1.

http://git.kernel.org/linus/e3ccaa9761200952cc269b1f4b7d7bb77a5e071b

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.5.0-9.9

---------------
linux (3.5.0-9.9) quantal-proposed; urgency=low

  [ Daniel P. Berrange ]

  * SAUCE: (drop after 3.6) Forbid invocation of kexec_load() outside
    initial PID namespace
    - LP: #1034125

  [ Douglas Bagnall ]

  * SAUCE: Unlock the rc_dev lock when the raw device is missing
    - LP: #1015836

  [ Ike Panhc ]

  * [Config] Enable EDAC/CLK for highbank
    - LP: #1008345

  [ Leann Ogasawara ]

  * Revert "ubuntu: AUFS -- reenable"

  [ Rob Herring ]

  * SAUCE: net: calxedaxgmac: add write barriers around setting owner bit
    - LP: #1008345
  * SAUCE: ARM smp_twd: add back "arm,smp-twd" compatible property
    - LP: #1008345
  * SAUCE: ARM: highbank: add soft power and reset key event handling
    - LP: #1008345
  * SAUCE: ARM: highbank: use writel_relaxed variant for pwr requests
    - LP: #1008345
  * SAUCE: ahci: un-staticize ahci_dev_classify
    - LP: #1008345
  * SAUCE: ahci_platform: add custom hard reset for Calxeda ahci ctrlr
    - LP: #1008345

  [ Upstream Kernel Changes ]

  * rt2x00: Add support for BUFFALO WLI-UC-GNM2 to rt2800usb.
    - LP: #871904
  * Avoid sysfs oops when an rc_dev's raw device is absent
    - LP: #1015836
  * eCryptfs: Copy up POSIX ACL and read-only flags from lower mount
  * clk: add DT clock binding support
    - LP: #1008345
  * clk: add DT fixed-clock binding support
    - LP: #1008345
  * clk: add highbank clock support
  * edac: add support for Calxeda highbank memory controller
    - LP: #1008345
  * edac: add support for Calxeda highbank L2 cache ecc
    - LP: #1008345
  * net: calxedaxgmac: enable rx cut-thru mode
    - LP: #1008345
  * net: calxedaxgmac: fix hang on rx refill
    - LP: #1008345
  * eCryptfs: Revert to a writethrough cache model
    - LP: #1034012
  * eCryptfs: Initialize empty lower files when opening them
    - LP: #911507
  * eCryptfs: Unlink lower inode when ecryptfs_create() fails
    - LP: #872905
 -- Leann Ogasawara <email address hidden> Wed, 08 Aug 2012 08:39:42 -0700

[Luis Henriques (henrix)]

This bug is awaiting verification that the kernel for Precise in -proposed solves the problem (3.2.0-30.47). Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-precise' to 'verification-done-precise'.

If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Colin Ian King (colin-king)]

Tested using lp:~colin-king/ecryptfs/test-fixes with ext2, ext3, ext4, xfs and btrfs lower file system, Linux ubuntu 3.0.0-25-server #41-Ubuntu SMP Mon Aug 13 18:18:27 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

sudo ./tests/run_tests.sh -K -c safe -b 1000000 -D /tmp/image -l /lower -u /upper -t lp-911507.sh -f ext2,ext3,ext4,xfs,btrfs
Running eCryptfs filesystem tests on ext2
lp-911507 pass
Running eCryptfs filesystem tests on ext3
lp-911507 pass
Running eCryptfs filesystem tests on ext4
lp-911507 pass
Running eCryptfs filesystem tests on xfs
lp-911507 pass
Running eCryptfs filesystem tests on btrfs
lp-911507 pass

Test Summary:
5 passed
0 failed

[Colin Ian King (colin-king)]

Tested using lp:~colin-king/ecryptfs/test-fixes with ext2, ext3, ext4, xfs and btrfs lower file system, Linux ubuntu 3.2.0-30-generic #47-Ubuntu SMP Wed Aug 15 19:30:10 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

sudo ./tests/run_tests.sh -K -c safe -b 1000000 -D /tmp/image -l /lower -u /upper -t lp-911507.sh -f ext2,ext3,ext4,xfs,btrfs
Running eCryptfs filesystem tests on ext2
lp-911507 pass
Running eCryptfs filesystem tests on ext3
lp-911507 pass
Running eCryptfs filesystem tests on ext4
lp-911507 pass
Running eCryptfs filesystem tests on xfs
lp-911507 pass
Running eCryptfs filesystem tests on btrfs
lp-911507 pass

Test Summary:
5 passed
0 failed

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.2.0-30.48

---------------
linux (3.2.0-30.48) precise-proposed; urgency=low

  [Luis Henriques]

  * Release Tracking Bug
    - LP: #1041217

  [ Upstream Kernel Changes ]

  * mutex: Place lock in contended state after fastpath_lock failure
    - LP: #1041114

linux (3.2.0-30.47) precise-proposed; urgency=low

  [Luis Henriques]

  * Release Tracking Bug
    - LP: #1036581

  [ Andy Whitcroft ]

  * add support for generating binary device trees and install them in
    /lib/firmware
    - LP: #1030600
  * [Config] add dtb_file configuration for highbank
    - LP: #1030600

  [ Chris Van Hoof ]

  * SAUCE: dell-laptop: additional rfkill blacklist Dell XPS 13
    - LP: #1030957
  * [Config] Add cifs support to the nfs-modules list
    - LP: #1031398

  [ Daniel P. Berrange ]

  * SAUCE: (drop after 3.6) Forbid invocation of kexec_load() outside
    initial PID namespace
    - LP: #1034125

  [ Dann Frazier ]

  * [Config] Compile the rtc-pl031 driver builtin on the highbank kernel
    flavour
    - LP: #1035110

  [ Douglas Bagnall ]

  * SAUCE: Unlock the rc_dev lock when the raw device is missing
    - LP: #1015836

  [ Rob Herring ]

  * SAUCE: ARM: highbank: add soft power and reset key event handling
    - LP: #1033853
  * SAUCE: ARM: highbank: use writel_relaxed variant for pwr requests
    - LP: #1033853
  * SAUCE: ahci: un-staticize ahci_dev_classify
    - LP: #1033853
  * SAUCE: ahci_platform: add custom hard reset for Calxeda ahci ctrlr
    - LP: #1033853

  [ Stefan Bader ]

  * (pre-stable) KVM: VMX: Set CPU_BASED_RDPMC_EXITING for nested
    - LP: #1031090

  [ Tim Gardner ]

  * [Config] updateconfigs

  [ Upstream Kernel Changes ]

  * ideapad: generate valid key event only
    - LP: #1029834
  * mm: reduce the amount of work done when updating min_free_kbytes
    - LP: #1032640
  * mm: compaction: allow compaction to isolate dirty pages
    - LP: #1032640
  * mm: compaction: determine if dirty pages can be migrated without
    blocking within ->migratepage
    - LP: #1032640
  * mm: page allocator: do not call direct reclaim for THP allocations
    while compaction is deferred
    - LP: #1032640
  * mm: compaction: make isolate_lru_page() filter-aware again
    - LP: #1032640
  * mm: compaction: introduce sync-light migration for use by compaction
    - LP: #1032640
  * mm: vmscan: when reclaiming for compaction, ensure there are sufficient
    free pages available
    - LP: #1032640
  * mm: vmscan: do not OOM if aborting reclaim to start compaction
    - LP: #1032640
  * mm: vmscan: check if reclaim should really abort even if
    compaction_ready() is true for one zone
    - LP: #1032640
  * vmscan: promote shared file mapped pages
    - LP: #1032640
  * vmscan: activate executable pages after first usage
    - LP: #1032640
  * mm/vmscan.c: consider swap space when deciding whether to continue
    reclaim
    - LP: #1032640
  * mm: test PageSwapBacked in lumpy reclaim
    - LP: #1032640
  * mm: vmscan: convert global reclaim to per-memcg LRU lists
    - LP: #1032640
  * cpuset: mm: reduce large amounts of memory barrier related damage v3
    - LP: #1032640
  * mm/hugetlb: fix warni...

[Chris Halse Rogers (raof)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.0.0-25.41

---------------
linux (3.0.0-25.41) oneiric-proposed; urgency=low

  [Luis Henriques]

  * Release Tracking Bug
    - LP: #1036178

  [ Andy Whitcroft ]

  * SAUCE: rds_ib_send() -- prevent local pings triggering BUG_ON()
    - LP: #1016299
    - CVE-2012-2372

  [ Douglas Bagnall ]

  * SAUCE: Unlock the rc_dev lock when the raw device is missing
    - LP: #1015836

  [ Upstream Kernel Changes ]

  * cifs: always update the inode cache with the results from a FIND_*
    - LP: #1031926
  * ntp: Fix STA_INS/DEL clearing bug
    - LP: #1031926
  * mm: fix lost kswapd wakeup in kswapd_stop()
    - LP: #1031926
  * MIPS: Properly align the .data..init_task section.
    - LP: #1031926
  * UBIFS: fix a bug in empty space fix-up
    - LP: #1031926
  * dm raid1: fix crash with mirror recovery and discard
    - LP: #1031926
  * mm/vmstat.c: cache align vm_stat
    - LP: #1031926
  * mm: memory hotplug: Check if pages are correctly reserved on a
    per-section basis
    - LP: #1031926
  * mm: reduce the amount of work done when updating min_free_kbytes
    - LP: #1031926
  * mm: vmscan: fix force-scanning small targets without swap
    - LP: #1031926
  * vmscan: clear ZONE_CONGESTED for zone with good watermark
    - LP: #1031926
  * vmscan: add shrink_slab tracepoints
    - LP: #1031926
  * vmscan: shrinker->nr updates race and go wrong
    - LP: #1031926
  * vmscan: reduce wind up shrinker->nr when shrinker can't do work
    - LP: #1031926
  * vmscan: limit direct reclaim for higher order allocations
    - LP: #1031926
  * vmscan: abort reclaim/compaction if compaction can proceed
    - LP: #1031926
  * mm: compaction: trivial clean up in acct_isolated()
    - LP: #1031926
  * mm: change isolate mode from #define to bitwise type
    - LP: #1031926
  * mm: compaction: make isolate_lru_page() filter-aware
    - LP: #1031926
  * mm: zone_reclaim: make isolate_lru_page() filter-aware
    - LP: #1031926
  * mm: migration: clean up unmap_and_move()
    - LP: #1031926
  * mm: compaction: allow compaction to isolate dirty pages
    - LP: #1031926
  * mm: compaction: determine if dirty pages can be migrated without
    blocking within ->migratepage
    - LP: #1031926
  * mm: page allocator: do not call direct reclaim for THP allocations
    while compaction is deferred
    - LP: #1031926
  * mm: compaction: make isolate_lru_page() filter-aware again
    - LP: #1031926
  * kswapd: avoid unnecessary rebalance after an unsuccessful balancing
    - LP: #1031926
  * kswapd: assign new_order and new_classzone_idx after wakeup in sleeping
    - LP: #1031926
  * mm: compaction: introduce sync-light migration for use by compaction
    - LP: #1031926
  * mm: vmscan: when reclaiming for compaction, ensure there are sufficient
    free pages available
    - LP: #1031926
  * mm: vmscan: do not OOM if aborting reclaim to start compaction
    - LP: #1031926
  * mm: vmscan: check if reclaim should really abort even if
    compaction_ready() is true for one zone
    - LP: #1031926
  * vmscan: promote shared file mapped pages
    - LP: #1031926
  * vmscan: activate executable pages after first usage
    - LP: #1031926
  *...

[John John Doe (johnjohndoe)]

I faced this error on Ubuntu 12.04. on my encrypted home partition. I noticed it when I tried to backup files with rsync. Also Firefox was broken somewhat. It could no longer load any bookmarks. I also had a full hard disc before which might have caused the problem. Now rsync seems to work - but there are still the error messages in the syslog - which is intended as fas as I understood. I wonder if someone could analyze the original problem that caused all those zero size files. I guess the initializer was my full disc. I leave this comment so that other users can find this bug report that came up with the same symptoms as I did. Thanx to the kernel crew!

[Colin Ian King (colin-king)]

Tested on Lucid Linux ubuntu 2.6.32-43-server #97-Ubuntu SMP Wed Sep 5 16:56:41 UTC 2012 x86_64 GNU/Linux

sudo ./tests/run_tests.sh -K -c safe -b 1000000 -D /tmp/image -l /lower -u /upper -t lp-911507.sh -f ext2,ext3,ext4,xfs,btrfs
Running eCryptfs filesystem tests on ext2
lp-911507 pass
Running eCryptfs filesystem tests on ext3
lp-911507 pass
Running eCryptfs filesystem tests on ext4
lp-911507 pass
Running eCryptfs filesystem tests on xfs
lp-911507 pass
Running eCryptfs filesystem tests on btrfs
lp-911507 pass

[rakkenes (sverre-rakkenes)]

I also have 12.04 and see many errors like these:

Sep 7 21:53:01 CCCx kernel: [ 1031.641254] Either the lower file is not in a valid eCryptfs format, or the key could not be r
etrieved. Plaintext passthrough mode is not enabled; returning -EIO
Sep 7 21:53:01 CCCux kernel: [ 1031.641340] Valid eCryptfs headers not found in file header region or xattr region, inode 8395
690

then my pc reboots, and I see these during boot.

Sep 7 21:56:35 CCux kernel: [ 10.858208] EXT4-fs (sdb1): ext4_orphan_cleanup: deleting unreferenced inode 8388879

not sure if this is what crashes my system, but the messages are the last things I see in my syslog each time I crash

[Luis Henriques (henrix)]

This bug is awaiting verification that the kernel for Natty in -proposed solves the problem (2.6.38-16.67). Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-natty' to 'verification-done-natty'.

If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Colin Ian King (colin-king)]

Tested on Natty -proposed: Linux ubuntu 2.6.38-16-server #67-Ubuntu SMP Thu Sep 6 18:15:24 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

sudo ./tests/run_tests.sh -K -c destructive -b 1000000 -D /tmp/image -l /lower -u /upper -t lp-911507.sh -f ext2,ext3,ext4,xfs,btrfs
Running eCryptfs filesystem tests on ext2
lp-911507 pass
Running eCryptfs filesystem tests on ext3
lp-911507 pass
Running eCryptfs filesystem tests on ext4
lp-911507 pass
Running eCryptfs filesystem tests on xfs
lp-911507 pass
Running eCryptfs filesystem tests on btrfs
lp-911507 pass

Test Summary:
5 passed
0 failed

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 2.6.38-16.67

---------------
linux (2.6.38-16.67) natty-proposed; urgency=low

  [Luis Henriques]

  * Release Tracking Bug
    - LP: #1045383

  [ Upstream Kernel Changes ]

  * rds: set correct msg_namelen
    - LP: #1031112
    - CVE-2012-3430
  * eCryptfs: Initialize empty lower files when opening them
    - LP: #911507
  * net: Allow driver to limit number of GSO segments per skb
    - LP: #1037456
    - CVE-2012-3412
  * tcp: do not scale TSO segment size with reordering degree
    - LP: #1037456
    - CVE-2012-3412
  * tcp: Apply device TSO segment limit earlier
    - LP: #1037456
    - CVE-2012-3412
  * sfc: Replace some literal constants with EFX_PAGE_SIZE/EFX_BUF_SIZE
    - LP: #1037456
    - CVE-2012-3412
  * sfc: Fix maximum number of TSO segments and minimum TX queue size
    - LP: #1037456
    - CVE-2012-3412
  * mm: Hold a file reference in madvise_remove
    - LP: #1042447
    - CVE-2012-3511
  * cred: copy_process() should clear child->replacement_session_keyring
    - LP: #1023535
    - CVE-2012-2745
 -- Luis Henriques <email address hidden> Thu, 06 Sep 2012 09:25:21 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 2.6.32-43.97

---------------
linux (2.6.32-43.97) lucid-proposed; urgency=low

  [Luis Henriques]

  * Release Tracking Bug
    - LP: #1045405

  [ Upstream Kernel Changes ]

  * rds: set correct msg_namelen
    - LP: #1031112
    - CVE-2012-3430
  * eCryptfs: Initialize empty lower files when opening them
    - LP: #911507
  * net: Allow driver to limit number of GSO segments per skb
    - LP: #1037456
    - CVE-2012-3412
  * tcp: do not scale TSO segment size with reordering degree
    - LP: #1037456
    - CVE-2012-3412
  * tcp: Apply device TSO segment limit earlier
    - LP: #1037456
    - CVE-2012-3412
  * sfc: Replace some literal constants with EFX_PAGE_SIZE/EFX_BUF_SIZE
    - LP: #1037456
    - CVE-2012-3412
  * sfc: Fix maximum number of TSO segments and minimum TX queue size
    - LP: #1037456
    - CVE-2012-3412
  * mm: Hold a file reference in madvise_remove
    - LP: #1042447
    - CVE-2012-3511
  * ulimit: raise default hard ulimit on number of files to 4096
    - LP: #663090
 -- Luis Henriques <email address hidden> Wed, 05 Sep 2012 09:39:41 +0100