ubuntu kernels < 3.7 do not include DISCARD/TRIM (FITRIM ioctl) for JFS filesystem

[Torsten Eichstädt]

I backported the (now uptream since linux 3.7) patch to add DISCARD/TRIM to JFS filesystem from
https://lkml.org/lkml/2012/10/3/250 to ubuntu kernel < 3.7. It contains two other minor fixes, too.

It's quite straightforward, the only change is to exclude one line (a call to dquot_writeback_dquots() in fs/jfs/super.c) that otherwise would pull in endless dependencies to newly added blkdev and quota functions and macros.

The patched 3.2 kernel (ubuntu generic 3.2.0-31.50, +JFS TRIM + LZO instead of GZIP, no other changes) is up and running on a SSD, 'fstrim' command works and also mount option 'discard'.

With the stock ubuntu 3.2 generic kernel on a SSD and JFS, I had two crashes, both inside JFS kernel code, one was catched (AppArmor?) leaving me with r/o filesystem and the other could not recover. Other than that, I experienced bad interactive performance since I installed the SSD. These have become slightly better with the patch added, but interactivity is still worse than with a rotational HD, e.g. when opening a menu. Will now investigate if running a low latency kernel fixes that.

The patch is appended, the Ubuntu KernelGitGuide says clone stock linux from kernel.ubuntu.com, but
https://wiki.ubuntu.com/Kernel/Dev/KernelPatches says clone stock linux-2.6 from kernel.org, that's all too confusing.
Decide what you want and adjust the docs, please... Now the patch is against ubuntu-3.2.

With kind regards, =|o)

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: linux-image-3.2.0-31-generic 3.2.0-31.50
ProcVersionSignature: Ubuntu 3.2.0-31.50-generic 3.2.28
Uname: Linux 3.2.28 x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 2.0.1-0ubuntu13
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: paul 2554 F.... pulseaudio
CRDA:
 country AM:
  (2402 - 2482 @ 40), (N/A, 20)
  (5170 - 5250 @ 20), (N/A, 18)
  (5250 - 5330 @ 20), (N/A, 18), DFS
Card0.Amixer.info:
 Card hw:0 'Intel'/'HDA Intel at 0xf0640000 irq 44'
   Mixer name : 'Realtek ALC262'
   Components : 'HDA:10ec0262,10cf0000,00100002 HDA:11c11040,11c10001,00100200'
   Controls : 29
   Simple ctrls : 16
Date: Mon Oct 8 14:45:44 2012
HibernationDevice: RESUME=UUID=42c86022-ee03-45af-bd1e-58be6d2aa822
InstallationMedia: Kubuntu 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120820.1)
Lsusb:
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: FUJITSU SIEMENS LIFEBOOK S7110
PccardctlIdent:
 Socket 0:
   no product info available
PccardctlStatus:
 Socket 0:
   no card
ProcEnviron:
 LANGUAGE=de_DE
 TERM=xterm-color
 PATH=(custom, user)
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.2.28 root=UUID=10e6b6ad-159a-41a1-8efe-04c1504aa075 ro crashkernel=384M-2G:64M,2G-:128M quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.2.28 N/A
 linux-backports-modules-3.2.28 N/A
 linux-firmware 1.79.1
RfKill:
 0: phy0: Wireless LAN
  Soft blocked: no
  Hard blocked: no
SourcePackage: linux
StagingDrivers: zram
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/10/2008
dmi.bios.vendor: FUJITSU // Phoenix Technologies Ltd.
dmi.bios.version: Version 1.35
dmi.board.name: FJNB1B5
dmi.board.vendor: FUJITSU
dmi.board.version: CP279611-02
dmi.chassis.type: 10
dmi.chassis.vendor: FUJITSU SIEMENS
dmi.chassis.version: S7110
dmi.modalias: dmi:bvnFUJITSU//PhoenixTechnologiesLtd.:bvrVersion1.35:bd09/10/2008:svnFUJITSUSIEMENS:pnLIFEBOOKS7110:pvr:rvnFUJITSU:rnFJNB1B5:rvrCP279611-02:cvnFUJITSUSIEMENS:ct10:cvrS7110:
dmi.product.name: LIFEBOOK S7110
dmi.sys.vendor: FUJITSU SIEMENS

[Torsten Eichstädt (torsten-eichstaedt)]

Update: With a low-latency kernel (3.2.28 aka ubuntu-3.2.0-31.50) responsiveness was better, but still much worse than on my old system w/ rotational HDD. Probably the problems were related to KDE/Kubuntu, because now, out of nowhere, the system is...
...FAAAAAST! _Everything_ is smooth. I don't know why. Installed 'preload', no change. Tweaked vm.swappiness=15 & vm.vfs_cache_pressure = 50, no change. Ah, wait, maybe it was discard=16 (default=64) in fstab for the JFS filesystem. But I can't think of that beeing the root of evil. Anyway, up to now I can tell the patch seems to be ok.
For the 3.5 kernel there is another call to dquot_writeback_blabla() that needs the same treatment. Other than that, git automagically merged it.

[Joseph Salisbury (jsalisbury)]

Thank you for providing a patch, and making Ubuntu better.

Can you provide some information on the status of the patch with regards to getting it merged upstream in linux-stable? Has it been sent upstream, what sort of feedback has it received, is it getting applied to a subsystem maintainer's tree, etc?

People affected by this bug are probably wondering why the kernel team doesn't just apply the patch and fix it. The reason is that the kernel team is reluctant (not opposed) to apply any patch to a stable kernel that is not from upstream. Applying patches that don't come from upstream add greatly to the support of the kernel as other upstream patches may touch the same area as the non-upstream patch and may prevent them from applying cleanly.

[Torsten Eichstädt (torsten-eichstaedt)]

Hello Joseph,

[short summary]
I do not follow kernel mailing lists. My infos are from news articles on the
internet and the web sites of Tino Reichard and lkml.org. I'm not familiar
with linux kernel interna and do not know who is the maintainer of JFS in
linux. I trust these, so I can derive:
1. The patch is upstream in linux 3.7
2. The patch is not sent upstream, it is available for git pull (see below).
3. feedback: It works for me; s/o with insight to linux disk quotas has to
evaluate if my change is ok if you run the patch on a system w/ quotas
enabled. From my limited understanding: In 3.2 and 3.5, there is no
dquot_writeback_dquots() and the patch does not alter other lines affecting
disk quotas -> I think it's ok to omit the call like I did. If not, one has
to do _much_ more work to "downgrade" the patch to 3.2/3.5 kernels.
Why not ask Timo or Dave or Wei Yongjun?

[unneccessarily long explanation]
I statet clearly in the initial bug description that I am not the author of
the patch, these are Tino Reichardt (TRIM), Dave Kleikamp & Wei Yongjun (minor
fixes, Dave seems to be kind of maintainer/coordinator) and that the info about
the patch is from https://lkml.org/lkml/2012/10/3/250. It is now upstream in
Linux 3.7 (news articles say that). All I did was puzzling how to apply
("downgrade") the patch from kernel 3.7 to kernel 3.2, and currently I'm doing
that for 3.5, but with low priority, because now I have a stable patched
ubuntu-3.2-low_latency kernel with TRIM on a SSD with JFS filesystem up and
running.
The status is (2nd update): *wonderfull*. It "Just Works (tm)".

Google is your friend, you can find some news articles and e-mails concerning
this subject (linux JFS TRIM DISCARD) very quickly.

I got the patch from git://github.com/kleikamp/linux-shaggy.git tags/jfs-3.7
and the linux source with git from kernel.ubuntu.com like described in
https://wiki.ubuntu.com/Kernel/Dev/KernelGitGuide:

$ git clone git://kernel.ubuntu.com/ubuntu/linux.git
$ git clone --reference linux git://kernel.ubuntu.com/ubuntu/ubuntu-
precise.git
and I set it to my current kernel (Kubuntu 12.04.1 with some backports-ppa):
$ git checkout -b Ubuntu-3.2.0-31.50+JFS_TRIM Ubuntu-3.2.0-31.50 (or .51?)

Then I applied the patch to find out the call to dquot_writeback_dquots() has
to be omited because it pulls in endless dependencies to things not in 3.2, so
I edited the patch manually, fixed rejections, etc. pp., and finally let diff
produce the patch to be shure it contains no typos. Now that I'm a little bit
more familiar with git, the next time I can do that with git for your
convenience.

With kind regards,
--
 =|o)

[Torsten Eichstädt (torsten-eichstaedt)]

In the meantime:
$ sudo adjust the ubuntu docs so it's clear for newbies to which kernel
version diff against (kernel.org/linux-meta/kernel.ubuntu.com)
and
$ sudo initiate to throw the debian so-called "helper" kernel-tools out of
ubuntu.

See my other bug filed concerning that. It took me a few hours to decide how
to alter the patch for 3.2 (because 1st I tried to pull in all the other stuff
as well, which is nonsense) and days to fight with 'make-kpkg' because I could
not have my kernel named 'ubuntu-3.2.0-31+JFS_TRIM', neither on the command
line nor in the .config file (CONFIG_LOCAL_VERSION) ...

> Hello Dave,
>
> I "downgraded" the latest JFS patch from
> git://github.com/kleikamp/linux-shaggy.git tags/jfs-3.7
> to linux 3.2 kernel. The only change is: I had to omit the call to
> dquot_writeback_dquots() in fs/jfs/super.c (search for HACK in the patch)
> to avoid pulling in endless dependencies to other new things.
> The same seems to hold true for 3.5 kernels, there is another call to
> dquot_writeback_dquots().
>
> Can you please verify that is ok or point me to the right direction
> otherwise?
>
> Thanks in advance,
--
 =|o)

[Torsten Eichstädt (torsten-eichstaedt)]

Hello Brad & Joseph,

I had a conversation with Tino Reichardt who wrote the TRIM parts of the
patch. Dave Kleikamp did not answer, maybe I choose the wrong e-mail adress
or because he went from Big Blue to Oracle?

Tino will support me to get the patch upstream, but he is "a small light" (and
I'm a nobody) on LKML, and he does not have the time for the all the e-mails
that need to be written and answered; but I want to try to do that. Do you
have contact with the maintainers of the stable Linux versions (and give
support)?

Can you provide a test environment for the stable Linux kernels when I send
you git-patches against bare Linux 2.6.34, 2.6.35, 3.0, 3.2, 3.4, 3.5 and 3.6
(i.e. a system with a SSD, maybe ssh access)? I can only reliably test my
version 3.2.28, and to be honest I'm running JFS because it's rock-solid and
do not want to gamble on my system... I could test in a virtual box, but
that's probably not sufficient?

Up to now I do not use disk quotas, that's the part that needs testing besides
usual stability tests. Is there a test suite I shall use?

Thanx in advance, =|o)

P.S. The patch is running on my system for weeks now very smooth and stable :)

Am Dienstag, 9. Oktober 2012, 15:46:59 schrieben Sie:
> Thank you for providing a patch, and making Ubuntu better.
>
> Can you provide some information on the status of the patch with regards
> to getting it merged upstream in linux-stable? Has it been sent
> upstream, what sort of feedback has it received, is it getting applied
> to a subsystem maintainer's tree, etc?
>
> People affected by this bug are probably wondering why the kernel team
> doesn't just apply the patch and fix it. The reason is that the kernel
> team is reluctant (not opposed) to apply any patch to a stable kernel
> that is not from upstream. Applying patches that don't come from
> upstream add greatly to the support of the kernel as other upstream
> patches may touch the same area as the non-upstream patch and may
> prevent them from applying cleanly.
>
> ** Changed in: linux (Ubuntu)
> Importance: Undecided => Medium
>
> ** Changed in: linux (Ubuntu)
> Status: Confirmed => Triaged
>
> ** Tags added: kernel-da-key
--
 =|o)

[Rolf Leggewie (r0lf)]

Thanks, guys, for your work. The only release still affected by this bug is precise which will be EOL by the end of this month. It's time to move on. Closing as obsolete.

[Rolf Leggewie (r0lf)]

The fix is in trusty and later, so setting status to fix released.