Trim doesn't work with lvm and dm-crypt

[dpslxoo]

I want to use fstrim together with lvm and dm-crypt but it doesn't work (see test procedure below). The problem seems to be somewhere in the lvm layer. Is there any fix or workaround for this?

My configuration:

`cat /etc/crypttab`

    sda3_crypt UUID=[...] none luks,discard

and

`cat /etc/lvm/lvm.conf`

    # [...]
    devices {
          # [ ... ]
          issue_discards = 1
          # [ ... ]
       }
    # [...]

The SSD is a Samsung 840 Pro.

Here is the output of `sudo dmsetup table`

    lubuntu--vg-root: 0 465903616 linear 252:0 2048
    lubuntu--vg-swap_1: 0 33308672 linear 252:0 465905664
    sda3_crypt: 0 499222528 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 8:3 4096 1 allow_discards

Here is my `/etc/fstab`:

    # <file system> <mount point> <type> <options> <dump> <pass>
    /dev/mapper/lubuntu--vg-root / ext4 errors=remount-ro 0 1
    # /boot was on /dev/sda2 during installation
    UUID=f700d855-96d0-495e-a480-81f52b965bda /boot ext2 defaults 0 2
    # /boot/efi was on /dev/sda1 during installation
    UUID=2296-2E49 /boot/efi vfat defaults 0 1
    /dev/mapper/lubuntu--vg-swap_1 none swap sw 0 0
    # tmp
    tmpfs /tmp tmpfs nodev,nosuid,noexec,mode=1777 0 0

Testing procedure:

yes | dd iflag=fullblock bs=1M count=1 of=trim.test

filefrag -s -v trim.test

Filesystem type is: ef53
File size of trim.test is 1048576 (256 blocks, blocksize 4096)
 ext logical physical expected length flags
   0 0 2224384 256 eof
trim.test: 1 extent found

df trim.test

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/lubuntu--vg-root 229164268 38651596 178865084 18% /

sudo dd bs=4096 skip=2224384 count=256 if=/dev/mapper/lubuntu--vg-root | hexdump -C

00000000 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a |y.y.y.y.y.y.y.y.|
*
256+0 records in
256+0 records out
1048576 bytes (1,0 MB) copied, 0,00328209 s, 319 MB/s
00100000

rm trim.test

sync

 sudo fstrim -v /
/: 695181312 bytes were trimmed

sync

sudo dd bs=4096 skip=2224384 count=256 if=/dev/mapper/lubuntu--vg-root | hexdump -C
00000000 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a |y.y.y.y.y.y.y.y.|
*
256+0 records in
256+0 records out
1048576 bytes (1,0 MB) copied, 0,00166157 s, 631 MB/s
00100000

I.e. the y pattern persists which indicates that TRIM doesn't work. See also my post on unix-stackexchange: http://unix.stackexchange.com/questions/85865/trim-with-lvm-and-dm-crypt

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: lvm2 2.02.95-6ubuntu4
ProcVersionSignature: Ubuntu 3.8.0-27.40-generic 3.8.13.4
Uname: Linux 3.8.0-27-generic x86_64
ApportVersion: 2.9.2-0ubuntu8.3
Architecture: amd64
Date: Sun Aug 18 14:31:46 2013
InstallationDate: Installed on 2013-08-03 (14 days ago)
InstallationMedia: Lubuntu 13.04 "Raring Ringtail" - Release amd64 (20130423.1)
MarkForUpload: True
SourcePackage: lvm2
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[dpslxoo (beagle-trash-mail)]

Any progress on this issue?

[frostschutz (frostschutz)]

It turns out to be a caching issue.

When a region of the SSD is trimmed, that data is lost. However for some unfathomable reason, Linux keeps the data in its caches. As a result, the 2nd dd call returns wrong data, making it seem as if the TRIM didn't work regardless whether it really did or not.

I am not sure if this is a kernel bug or not. It should not have any ill effects in practice (after all who usually tries to read from a freshly TRIMmed region?) but even so it seems like a waste of cache memory if it's occupied by invalidated data.

Anyway, drop caches before issuing the dd command.

    echo 1 | sudo tee /proc/sys/vm/drop_caches

    sudo dd bs=4096 skip=2224384 count=256 if=/dev/mapper/lubuntu--vg-root | hexdump -C

Then see if it still returns y.y.y. or not. If it does not - trim has been working as it is supposed to be.

Also the issue_discards in LVM is not strictly necessary, it only affects lvremove, lvreduce, etc. Filesystem based TRIM seem to be passed on by LVM unconditionally.