ecryptfs corrupts files over 4GB size on i686

Bug #1243636 reported by Lars Düsing on 2013-10-23
36
This bug affects 5 people
Affects Status Importance Assigned to Milestone
eCryptfs
Critical
Tyler Hicks
linux (Ubuntu)
Critical
Tyler Hicks

Bug Description

[SRU Justification]

Commit 24d15266bd86b7961f309a962fa3aa177a78c49f introduced a data corruption
regression on 32 bit architectures when writing past the 4 GB.

[Impact]

32 bit users experience corruption of large files.

[Fix]

A cast is needed when shifting the page's index. Colin and I independently
identified the problem. It is a simple fix that has been merged upstream:

http://git.kernel.org/linus/43b7c6c6a4e3916edd186ceb61be0c67d1e0969e

[Test Case]

Inside of an eCryptfs mount on an i686 Ubuntu install, create a file containing
4 GB + 1 page worth (4096 bytes) of zeros. Then inspect the file for non-zero
bytes.

$ rm zeros
$ dd if=/dev/zero of=zeros bs=4096 count=$((4*1024*1024*1024/4096+4096))
1052672+0 records in
1052672+0 records out
4311744512 bytes (4.3 GB) copied, 226.133 s, 19.1 MB/s
$ hexdump -C zeros
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
101000000

The hexdump output should show all zeros. A non patched kernel will show
non-zero bytes.

[Original Bug Report]

on extracting files with extracted size >4 GB files are getting currupted.
interestingly file gets currupted in the very moment the file size gets more than 4GB.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: linux-image-3.11.0-12-generic 3.11.0-12.19
ProcVersionSignature: Ubuntu 3.11.0-12.19-generic 3.11.3
Uname: Linux 3.11.0-12-generic i686
ApportVersion: 2.12.5-0ubuntu2
Architecture: i386
Date: Wed Oct 23 12:11:43 2013
EcryptfsInUse: Yes
InstallationDate: Installed on 2013-07-20 (94 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Release i386 (20130424)
MarkForUpload: True
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.11.0-12-generic root=UUID=e97431f7-60b7-4fbe-b22f-5ca3304f2d50 ro quiet splash vt.handoff=7
SourcePackage: linux
UpgradeStatus: Upgraded to saucy on 2013-09-08 (45 days ago)

Lars Düsing (lars.duesing) wrote :

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed

I can reproduce this in saucy and on mainline (3.12.0-031200rc6-generic). It is specific to i686.

My guess is that a size_t is being used when a loff_t is needed. I'm looking into it now.

Changed in linux (Ubuntu):
assignee: nobody → Tyler Hicks (tyhicks)
importance: Undecided → Critical
Joseph Salisbury (jsalisbury) wrote :

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.12 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-rc6-saucy/

Changed in linux (Ubuntu):
importance: Critical → Medium
status: Confirmed → Incomplete
importance: Medium → High
Joseph Salisbury (jsalisbury) wrote :

Disregard my comment in #4. Commented at almost the same time :-)

Tyler Hicks (tyhicks) on 2013-10-23
tags: added: kernel-bug-exists-upstream
Changed in ecryptfs:
status: New → Triaged
Changed in linux (Ubuntu):
status: Incomplete → Triaged
Changed in ecryptfs:
importance: Undecided → Critical
assignee: nobody → Tyler Hicks (tyhicks)
Changed in linux (Ubuntu):
importance: High → Critical
tags: added: kernel-da-key
tags: added: kernel-key
removed: kernel-da-key
Tyler Hicks (tyhicks) on 2013-10-24
summary: - ecryptfs currupts files over 4GB size
+ ecryptfs currupts files over 4GB size on i686
Changed in ecryptfs:
status: Triaged → In Progress
Changed in linux (Ubuntu):
status: Triaged → In Progress

Tyler, the attached diff helps me read/write files over 4GB. I think some of the changes may be redundant, so I'm checking to see what is the minimal set of changes required.

Colin Ian King (colin-king) wrote :

Tyler, reduced it down to a one line fix. Tested with a 0.5GB, 2GB, 4GB, 8GB, 32GB test file with a sanity check on each byte. Patch attached below. Not yet tested with ecryptfs tests, but it looks like the root issue.

tags: added: tasty
tags: added: patch
Tyler Hicks (tyhicks) wrote :

Thanks Colin!

As I mentioned in IRC, I came up with the same patch last night and successfully ran tests against it overnight. We'll go with your patch since it already has a commit message. I targeted it for 3.11+ stable and will push it to Linus today. I'll also get the Ubuntu SRU prepared.

Tyler Hicks (tyhicks) on 2013-10-24
description: updated
Lars Düsing (lars.duesing) wrote :

Patch works for me on 3.12-rc6.

Thanks Colin and Tyler!

summary: - ecryptfs currupts files over 4GB size on i686
+ ecryptfs corrupts files over 4GB size on i686
Tyler Hicks (tyhicks) on 2013-10-25
description: updated
Changed in ecryptfs:
status: In Progress → Fix Committed
striscio (gianpaolo-racca) wrote :

Any update on this? I would like to use encrypted home, but I need to use big files for virtualbox

Lars Düsing (lars.duesing) wrote :

striscio, problem has been fixed on mainline-kernel 3.12, if you have to, just update like described in: http://ubuntuhandbook.org/index.php/2013/11/linux-kernel-3-12-released-install-ubuntu-or-linux-mint/
update to saucy-kernel will be as soon as possible.

tags: removed: kernel-key
Brad Figg (brad-figg) on 2013-11-14
Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-saucy' to 'verification-done-saucy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-saucy
Lars Düsing (lars.duesing) wrote :

-proposed works for me. Tested against NUL-File and multiple VirtualBox-images.

tags: added: verification-done-saucy
removed: verification-needed-saucy
Launchpad Janitor (janitor) wrote :
Download full text (5.6 KiB)

This bug was fixed in the package linux - 3.11.0-14.21

---------------
linux (3.11.0-14.21) saucy; urgency=low

  [Brad Figg]

  * Release Tracking Bug
    - LP: #1250540

  [ Anthony Wong ]

  * SAUCE: Work around broken ACPI backlight on Dell Inspiron 5537
    - LP: #1231305

  [ Colin Ian King ]

  * SAUCE: eCryptfs: fix 32 bit corruption issue
    - LP: #1243636

  [ Ming Lei ]

  * SAUCE: ext4: fix performance regression in ext4_writepages
    - LP: #1242812

  [ Upstream Kernel Changes ]

  * Revert "bridge: only expire the mdb entry when query is received"
    - LP: #1249081
  * ext4: fix performance regression in writeback of random writes
    - LP: #1242812
  * be2net: pass if_id for v1 and V2 versions of TX_CREATE cmd
    - LP: #1234019
  * tcp: TSO packets automatic sizing
    - LP: #1249081
  * tcp: TSQ can use a dynamic limit
    - LP: #1249081
  * tcp: must unclone packets before mangling them
    - LP: #1249081
  * tcp: do not forget FIN in tcp_shifted_skb()
    - LP: #1249081
  * tcp: fix incorrect ca_state in tail loss probe
    - LP: #1249081
  * net: do not call sock_put() on TIMEWAIT sockets
    - LP: #1249081
  * batman-adv: set up network coding packet handlers during module init
    - LP: #1249081
  * l2tp: fix kernel panic when using IPv4-mapped IPv6 addresses
    - LP: #1249081
  * l2tp: Fix build warning with ipv6 disabled.
    - LP: #1249081
  * net: mv643xx_eth: update statistics timer from timer context only
    - LP: #1249081
  * net: mv643xx_eth: fix orphaned statistics timer crash
    - LP: #1249081
  * net: heap overflow in __audit_sockaddr()
    - LP: #1249081
  * sit: amend "allow to use rtnl ops on fb tunnel"
    - LP: #1249081
  * proc connector: fix info leaks
    - LP: #1249081
  * ipv4: fix ineffective source address selection
    - LP: #1249081
  * can: dev: fix nlmsg size calculation in can_get_size()
    - LP: #1249081
  * net: secure_seq: Fix warning when CONFIG_IPV6 and CONFIG_INET are not
    selected
    - LP: #1249081
  * xen-netback: Don't destroy the netdev until the vif is shut down
    - LP: #1249081
  * net/mlx4_en: Rename name of mlx4_en_rx_alloc members
    - LP: #1249081
  * net/mlx4_en: Fix pages never dma unmapped on rx
    - LP: #1249081
  * net: vlan: fix nlmsg size calculation in vlan_get_size()
    - LP: #1249081
  * bridge: update mdb expiration timer upon reports.
    - LP: #1249081
  * vti: get rid of nf mark rule in prerouting
    - LP: #1249081
  * l2tp: must disable bh before calling l2tp_xmit_skb()
    - LP: #1249081
  * netem: update backlog after drop
    - LP: #1249081
  * netem: free skb's in tree on reset
    - LP: #1249081
  * farsync: fix info leak in ioctl
    - LP: #1249081
  * unix_diag: fix info leak
    - LP: #1249081
  * connector: use nlmsg_len() to check message length
    - LP: #1249081
  * bnx2x: record rx queue for LRO packets
    - LP: #1249081
  * virtio-net: don't respond to cpu hotplug notifier if we're not ready
    - LP: #1249081
  * virtio-net: refill only when device is up during setting queues
    - LP: #1249081
  * bridge: Correctly clamp MAX forward_delay when enabling STP
    - LP: #1249081
  * net: dst: provide accessor function to dst->xfrm
 ...

Read more...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in ecryptfs:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers