add NX 842 hw compression patches

[bugproxy]

[Impact]
Users of PowerPC processors with an NX coprocessor will not be able to use features like memory compression without this set of patches.

[Test Case]
Run kernel on a PowerPC processor with an NX coprocessor implementing 842 memory compression.

[Fix]

Commits:
b130e7c04f1130f241eddf72098a6e08d41d08fe
edc424f8cd84bbae530d8a9f86caf1923606fb24
2da572c959dd5815aef153cf62010b16a498a0d3
2062c5b6da758ca2bb64b698d7f9c5c45a06fcf9
fdd05e4b9ae22603ed09beb4e179ae7746555a81
7011a122383e36dab594406720fa1d089e0be8f9
959e6659b6f74ec1fa4d391a3b88d63dc0189f36
99182a42b7ef3d5e4180992ce01befd9e87526d2
b8e04187c90107c58d1ccbeb68a0ba4c5bfd4167
ed70b479c2c0b6e1319f0cb2de19f1051be219a4
f7ead7b47a758bbee6fdc66f95f27fdb866e5e9d
ca7fc7e962fa067ba31f76a6e5828537394f6481
9358eac06b8fcf9fcaf566ec825fc05e2b1acd10
3e648cbeb31be5cb84b9ec19822e2b85417f07c4
3154de71258a32040214fda174e67b975b0810ef
32be6d3e362b896c81aae7c635d44e5a91406ce2
2c6f6eabc0bfcea0a62370038da713e3873cff31
c47d63020c03659e584673f78f24f2e5de3e6b9b

Plus the following configuration changes:

+CONFIG_842_COMPRESS=m
+CONFIG_842_DECOMPRESS=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=m

In addition some modules need to be loaded by initramfs which requires changes to the initramfs-tools package.

The modules are:
nx-compress
nx-compress-crypto
nx-compress-platform
nx-compress-pseries
nx-compress-powernv
842-decompress

--

Please add/include the patches for PowerPC NX coprocessor 842 memory compression. They're currently in Herbert Xu's upstream tree, and presumably he will send them up to Linus at the 4.2 merge window.

You can find the commits at
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git

ed70b47 crypto: nx - add hardware 842 crypto comp alg
b8e04187 crypto: nx - simplify pSeries nx842 driver
99182a42 crypto: nx - add PowerNV platform NX-842 driver
959e665 crypto: nx - add nx842 constraints
7011a12 crypto: nx - add NX-842 platform frontend driver
fdd05e4 crypto: nx - rename nx-842.c to nx-842-pseries.c
2062c5b crypto: 842 - change 842 alg to use software
2da572c lib: add software 842 compression/decompression
edc424f powerpc: Add ICSWX instruction
b130e7c powerpc: export of_get_ibm_chip_id function

I'll update this when they are added to Linus' tree.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1454687/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Chris J Arges (arges)]

Waiting on these to land in Linus' tree.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-06-23 11:02 EDT-------
Patches are in Linus' tree:

ed70b47 crypto: nx - add hardware 842 crypto comp alg
b8e04187 crypto: nx - simplify pSeries nx842 driver
99182a42 crypto: nx - add PowerNV platform NX-842 driver
959e665 crypto: nx - add nx842 constraints
7011a12 crypto: nx - add NX-842 platform frontend driver
fdd05e4 crypto: nx - rename nx-842.c to nx-842-pseries.c
2062c5b crypto: 842 - change 842 alg to use software
2da572c lib: add software 842 compression/decompression
edc424f powerpc: Add ICSWX instruction
b130e7c powerpc: export of_get_ibm_chip_id function

additionally, these patches further improve NX-842 and are also in Linus' tree:

c47d630 crypto: nx - add LE support to pSeries platform driver
2c6f6ea crypto: nx - replace NX842_MEM_COMPRESS with function
32be6d3 crypto: nx - move include/linux/nx842.h into drivers/crypto/nx/nx-842.h
3154de7 crypto: nx - fix nx-842 pSeries driver minimum buffer size
3e648cb crypto: nx - prevent nx 842 load if no hw driver
9358eac crypto: nx - remove 842-nx null checks
ca7fc7e lib: correct 842 decompress for 32 bit
f7ead7b lib: make lib/842 decompress functions static

------- Comment From <email address hidden> 2015-06-23 16:17 EDT-------
Also, to enable NX-842 and zswap (which is the primary user of NX-842 in our case), please set:

CONFIG_CRYPTO_DEV_NX=y
CONFIG_CRYPTO_DEV_NX_COMPRESS=y
CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=y
CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=y
CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_842_DECOMPRESS=y
CONFIG_ZSWAP=y
CONFIG_ZPOOL=y
CONFIG_ZBUD=y
CONFIG_ZSMALLOC=y

[Chris J Arges (arges)]

All patches cleanly cherry-pick into 3.19.

The following config options are already set in Ubuntu:

CONFIG_ZSWAP=y
CONFIG_ZPOOL=y
CONFIG_ZBUD=y
CONFIG_ZSMALLOC=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_DEV_NX=y
CONFIG_CRYPTO_DEV_NX_COMPRESS=m

I'll add the following:
+CONFIG_842_COMPRESS=m
+CONFIG_842_DECOMPRESS=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=m

The only changes are build these as modules, and add 842_COMPRESS.

Does this work?

Thanks,

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-06-25 13:13 EDT-------
(In reply to comment #11)
> All patches cleanly cherry-pick into 3.19.
>
> The following config options are already set in Ubuntu:
>
> CONFIG_ZSWAP=y
> CONFIG_ZPOOL=y
> CONFIG_ZBUD=y
> CONFIG_ZSMALLOC=y
> CONFIG_CRYPTO_ALGAPI=y
> CONFIG_CRYPTO_DEV_NX=y
> CONFIG_CRYPTO_DEV_NX_COMPRESS=m
>
> I'll add the following:
> +CONFIG_842_COMPRESS=m
> +CONFIG_842_DECOMPRESS=m
> +CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=m
> +CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=m
> +CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=m
>
> The only changes are build these as modules, and add 842_COMPRESS.
>
> Does this work?

yep that's fine.

The downside of them being modules is that to use NX 842 with zswap - which requires configuration to all be done at boot time (currently, I have patches pending to change that) - the initramfs has to be rebuilt with the NX 842 modules. But that's not a big deal, just creating a dracut.d conf file and rerunning dracut.

[Chris J Arges (arges)]

In Ubuntu we use initramfs-tools to update the initramfs. In this package there is a 'auto_add_modules' hook-function. So we can add the modules here. Can you give me a specific list of which modules are required?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-06-25 18:38 EDT-------
(In reply to comment #13)
> In Ubuntu we use initramfs-tools to update the initramfs.

ah ok.

> In this package
> there is a 'auto_add_modules' hook-function. So we can add the modules
> here. Can you give me a specific list of which modules are required?

Assuming it's ok that the modules won't always be present - i.e. they're only built on powerpc - the modules will be:

nx-compress
nx-compress-crypto
nx-compress-platform
nx-compress-pseries
nx-compress-powernv
842-decompress

the 842-compress module actually isn't needed; the 842-compress and 842-decompress modules are only the "reference implementation" in software, and due to the specifics of the 842 format they are very painfully slow; 842 is really only suited to be done in custom hardware.

The 842-decompress module is needed, because the 842 hardware driver falls back to using it if the hardware fails or is disabled, so that decompression never fails.

but including the 842-compress module doesn't hurt anything.

[Chris J Arges (arges)]

You can review my pull request here:
git://kernel.ubuntu.com/arges/ubuntu-vivid.git lp1454687

Working on patches for initramfs.

[Chris J Arges (arges)]

Attached is a patch to initramfs for wily. Once the kernel patches are in I'll build a test package for ppc64el.
Is there an easy way to test functionality? And can I do this on a power8/powernv system?

[Launchpad Janitor (janitor)]

This bug was fixed in the package initramfs-tools - 0.103ubuntu16

---------------
initramfs-tools (0.103ubuntu16) wily; urgency=medium

  * hook-functions: Add nx-* and 842-* modules in order to enable PowerPC NX
    Crypto Coprocessor. (LP: #1454687)

 -- Chris J Arges <email address hidden> Thu, 25 Jun 2015 13:59:55 -0500

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-vivid' to 'verification-done-vivid'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.19.0-23.24

---------------
linux (3.19.0-23.24) vivid; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1472346

  [ Chris J Arges ]

  * SAUCE: Don't use atomic read in evlist.c
    - LP: #1410673

linux (3.19.0-23.23) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1472048

  [ Chris J Arges ]

  * [Config] Add CRYPTO_DEV_NX_*, 842_* as modules
    - LP: #1454687

  [ Lu, Han ]

  * SAUCE: i915_bpo: drm/i915/audio: add codec wakeup override
    enabled/disable callback
    - LP: #1460674

  [ Timo Aaltonen ]

  * SAUCE: Backport I915_OVERLAY_DISABLE_DEST_COLORKEY
    - LP: #1460674
  * SAUCE: i915_bpo: Rebase to drm-intel-next-fixes-2015-05-29
    - LP: #1460674
  * SAUCE: i915_bpo: Revert "drm/i915: Implement the intel_dp_autotest_edid
    function for DP EDID complaince tests"
    - LP: #1460674
  * SAUCE: i915_bpo: Revert "drm/i915: Add debugfs test control files for
    Displayport compliance testing"
    - LP: #1460674
  * SAUCE: Load i915_bpo from the hda driver on SKL/CHV
    - LP: #1460674
  * SAUCE: i915_bpo: Don't try to support BXT
    - LP: #1460674
  * SAUCE: i915_bpo: drm/i915/skl: Fix DMC API version.

  [ Upstream Kernel Changes ]

  * Revert "usb: dwc2: add bus suspend/resume for dwc2"
    - LP: #1471252
  * Revert "HID: logitech-hidpp: support combo keyboard touchpad TK820"
    - LP: #1471252
  * Revert "KVM: x86: drop fpu_activate hook"
    - LP: #1471252
  * Revert "libceph: clear r_req_lru_item in __unregister_linger_request()"
    - LP: #1471252
  * drm/i915: add component support
    - LP: #1460661
  * ALSA: hda: export struct hda_intel
    - LP: #1460661
  * ALSA: hda: pass intel_hda to all i915 interface functions
    - LP: #1460661
  * ALSA: hda: add component support
    - LP: #1460661
  * drm/atomic-helpers: Fix documentation typos and wrong copy&paste
    - LP: #1460674
  * drm/atomic: Rename drm_atomic_helper_commit_pre_planes() state argument
    - LP: #1460674
  * drm/atomic-helper: Rename commmit_post/pre_planes
    - LP: #1460674
  * drm/atomic-helpers: make mode_set hooks optional
    - LP: #1460674
  * drm/atomic-helper: Fix kerneldoc for prepare_planes
    - LP: #1460674
  * drm: Complete moving rotation property to core
    - LP: #1460674
  * drm: Share plane pixel format check code between legacy and atomic
    - LP: #1460674
  * drm/atomic: Constify a bunch of functions pointer structs
    - LP: #1460674
  * drm: Fix some typo mistake of the annotations
    - LP: #1460674
  * drm: change connector to tmp_connector
    - LP: #1460674
  * drm: atomic: Expose CRTC active property
    - LP: #1460674
  * drm: atomic: Allow setting CRTC active property
    - LP: #1460674
  * drm/atomic-helpers: Properly avoid full modeset dance
    - LP: #1460674
  * drm/atomic: Add helpers for state-subclassing drivers
    - LP: #1460674
  * drm: Fix some typos
    - LP: #1460674
  * drm/atomic: Add for_each_{connector,crtc,plane}_in_state helper macros
    - LP: #1460674
  * drm/atomic-helper: Don't call atomic_update_plane when it stays off
    - LP: #1460674
  * drm/atomic-helper: Really recover pre-atomic plane/cursor behavior
 ...

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-07-23 16:25 EDT-------
There are actually some more NX 842 patches that should be in the 4.3 kernel. They're currently all in Herbert's tree:
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git

In the usual descending order:

03952d9 crypto: nx - make platform drivers directly register with crypto
174d66d crypto: nx - rename nx-842-crypto.c to nx-842.c
d31581a crypto: nx - merge nx-compress and nx-compress-crypto
20fc311 crypto: nx - use common code for both NX decompress success cases
ee781b7 crypto: nx - don't register pSeries driver if ENODEV
7f6e3aa crypto: nx - move kzalloc() out of spinlock
90fd73f crypto: nx - remove pSeries NX 'status' field
039af96 crypto: nx - remove __init/__exit from VIO functions
23ad69a crypto: nx/842 - Fix context corruption
2b93f7e crypto: nx - reduce chattiness of platform drivers
7abd75b crypto: nx - do not emit extra output if status is disabled
ec13bcb crypto: nx - rename nx842_{init, exit} to nx842_pseries_{init, exit}
fa9a9a0 crypto: nx - nx842_OF_upd_status should return ENODEV if device is not 'okay'

Should I open a new bug for these additional patches, or can you add them using this bug?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-07-29 20:03 EDT-------
> Should I open a new bug for these additional patches, or can you add them
> using this bug?
Please reopen a new bug for this request, since the current bug is closed on Canonical side and this one should be closed also.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-08-27 13:22 EDT-------
> Haren, you may want to follow up with a new Ubuntu bug with the patches from
> comment 21.
Haren, did you opened a new feature for the inclusion of the other patches?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-08-27 15:41 EDT-------
Hello. Haren has opened it and it was mirrored to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1488495