RPT related fixes missing in Ubuntu 16.04.3

[bugproxy]

== SRU Justification ==
These four commits are Radix performance patches specific to powerpc.

The patches are reported to help anywhere from 40% up to 3.5X on unixbench fork/exec testcases.

The first patch was included in mainlne as of 4.12-rc1. The last three patches are
still in linux-next.

---Problem Description---

The following patches are missing in the ubuntu 16.04.3 kernel

https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-April/156104.html

https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-April/156105.html

https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-April/157130.html

---uname output---

4.10.0-29-generic #33~16.04.1-Ubuntu SMP Tue Jul 25 18:17:06 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = power9

---Steps to Reproduce---

Did a fork and exec benchmark to look at the performance and they pointed to radix routines.

Upstream commits mentioned:

commit f7327e0ba3805470cced2acfa053e795362ee41d

powerpc/mm/radix: Remove unnecessary ptesync

commit f6b0df55cad252fedd60aa2ba75a0207d0006283

powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush

commit a5998fcb92552a18713b6aa5c146aa400e4d75ee

powerpc/mm/radix: Optimise tlbiel flush all case

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-08-08 11:39 EDT-------
(In reply to comment #4)
Looking at the "Ubuntu-hwe-4.10.0-30.34_16.04.1" branch, it appears the first two are included:

> commit f7327e0ba3805470cced2acfa053e795362ee41d
> powerpc/mm/radix: Remove unnecessary ptesync
commit 2da0c6d3eb566166126b7a4180ac2a10729ae034
Author: Aneesh Kumar K.V <email address hidden>
Date: Mon Apr 10 10:23:46 2017 -0300

UBUNTU: SAUCE: powerpc/mm/radix: Remove unnecessary ptesync
BugLink: https://bugs.launchpad.net/bugs/1681429

>
> commit f6b0df55cad252fedd60aa2ba75a0207d0006283
> powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush
>
commit ea856a8baccee7dd3fd9fd37a37cd688d534a0b6
Author: Aneesh Kumar K.V <email address hidden>
Date: Mon Apr 10 10:23:45 2017 -0300

UBUNTU: SAUCE: powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush
BugLink: https://bugs.launchpad.net/bugs/1681429

** But this one is missing:

> commit a5998fcb92552a18713b6aa5c146aa400e4d75ee
> powerpc/mm/radix: Optimise tlbiel flush all case

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-08-08 15:30 EDT-------
So, this bug is basically a request to include a5998fcb92552a18713b6aa5c146aa400e4d75ee into Ubuntu Zesty (4.10) kernel, correct?

------- Comment From <email address hidden> 2017-08-08 15:36 EDT-------
(In reply to comment #6)
> So, this bug is basically a request to include
> a5998fcb92552a18713b6aa5c146aa400e4d75ee into Ubuntu Zesty (4.10) kernel,
> correct?

Yep! would like this fix for the 16.04.3 hwe 4.10 kernel

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-08-10 10:41 EDT-------
Aneesh has additional Radix performance fixes (3 more total):

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=424de9c6e3f89399fc11afc1f53f89c5329132da
("powerpc/mm/radix: Avoid flushing the PWC on every flush_tlb_range")
.
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=a46cc7a90fd8d95bfbb2b27080efe872a1a51db4
("powerpc/mm/radix: Improve TLB/PWC flushes")
.
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=5ce5fe14ed0302315061cf97ce67accd1b25b938
("powerpc/mm/radix: Improve _tlbiel_pid to be usable for PWC flushes")

(upstream, currently scheduled for 4.14)

Can these be included in Zesty as well?

FYI the first set of 3 patches mentioned in the orig bug report helps anywhere from 40% up to 3.5X on unixbench fork/exec testcases. I will post updated numbers for this second set of 3 fixes soon.

[Joseph Salisbury (jsalisbury)]

The first two commits listed in the bug description are indeed already in Zesty via bug 1681429.

I built a test kernel with the third of the requested patches in the description(Commit a5998fcb92). This test kernel also has the three additional patches requested in comment #3.

The first and third patches mentioned comment #3(5ce5fe14ed and 424de9c6e3) picked cleanly from linux-next. However, commit a46cc7a90f needed a minor backport.

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1709220/

Can you test this kernel and see if it resolves this bug?

Thanks in advance!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-08-15 12:06 EDT-------
Thanks! Seeing good improvements with this build based on some unixbench microbenchmarks:

baseline = 4.10.0-32-generic
RPT fixes = 4.10.0-32.36~lp1709220

Unixbench baseline RPT fixes improvement
SMToff SMT4 SMToff SMT4 SMToff SMT4
Execl ? single 656.2 963.0 1583.0 1507.1 141.2% 56.5%
Execl ? multi 2522.9 6527.3 4720.2 11766.7 87.1% 80.3%
Sysexec ? single 487.2 715.5 787.0 1082.2 61.5% 51.3%
Sysexec ? multi 1783.0 4845.2 2738.6 8029.3 53.6% 65.7%
Spawn ? single 1645.0 2546.8 2632.9 3168.8 60.1% 24.4%
Spawn ? multi 5877.5 17688.8 8459.9 23650.1 43.9% 33.7%

Note this was on a 4core DD1 system running at 1.6GHz.

[Kleber Sacilotto de Souza (kleber-souza)]

The fixes were applied to zesty/master-next branch and will be available on the next kernel SRU cycle.

I removed the verification-done-zesty tag because it must be added only after the fixes have been tested with a kernel that's at least in -proposed. So the verification will need to be done again when the next SRU kernel hits -proposed (a comment will be added when that happens).

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-08-24 12:41 EDT-------
(In reply to comment #15)
> The fixes were applied to zesty/master-next branch and will be available on
> the next kernel SRU cycle.
>
> I removed the verification-done-zesty tag because it must be added only
> after the fixes have been tested with a kernel that's at least in -proposed.
> So the verification will need to be done again when the next SRU kernel hits
> -proposed (a comment will be added when that happens).

Oops sorry, got too excited about verification ;) Thanks for the info.

[Kleber Sacilotto de Souza (kleber-souza)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-zesty' to 'verification-done-zesty'. If the problem still exists, change the tag 'verification-needed-zesty' to 'verification-failed-zesty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Kleber Sacilotto de Souza (kleber-souza)]

Hello IBM,

Could you please verify the fix with the lasted Zesty kernel in -proposed?

Thank you!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-07 16:14 EDT-------
Hi, I was able to verify that these fixes in the -proposed kernel provide the expected performance improvements, thanks!

This time I re-baselined on a P9 with 32 DD2 cores, so overall compare is a bit different than previous comments but still seeing significant performance gains as expected. Here are the improvements of the proposed kernel (4.10.0-34-generic -- #38~16.04.1-Ubuntu) compared to the baseline kernel (4.10.0-33-generic #37~16.04.1-Ubuntu):

improvement
SMToff SMT4
Execl ? single 136.6% 153.0%
Execl ? multi -6.2% 8.8%
Sysexec ? single 110.1% 91.1%
Sysexec ? multi 2.8% 5.6%
Spawn ? single 43.9% 46.1%
Spawn ? multi 8.3% 14.2%

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.12.0-13.14

---------------
linux (4.12.0-13.14) artful; urgency=low

  * linux: 4.12.0-13.14 -proposed tracker (LP: #1714687)

  * vhost guest network randomly drops under stress (kvm) (LP: #1711251)
    - Revert "vhost: cache used event for better performance"

  * EDAC sbridge: Failed to register device with error -22. (LP: #1714112)
    - [Config] CONFIG_EDAC_GHES=n

  * Artful update to v4.12.10 stable release (LP: #1714525)
    - sparc64: remove unnecessary log message
    - bonding: require speed/duplex only for 802.3ad, alb and tlb
    - bonding: ratelimit failed speed/duplex update warning
    - af_key: do not use GFP_KERNEL in atomic contexts
    - dccp: purge write queue in dccp_destroy_sock()
    - dccp: defer ccid_hc_tx_delete() at dismantle time
    - ipv4: fix NULL dereference in free_fib_info_rcu()
    - net_sched/sfq: update hierarchical backlog when drop packet
    - net_sched: remove warning from qdisc_hash_add
    - bpf: fix bpf_trace_printk on 32 bit archs
    - net: igmp: Use ingress interface rather than vrf device
    - openvswitch: fix skb_panic due to the incorrect actions attrlen
    - ptr_ring: use kmalloc_array()
    - ipv4: better IP_MAX_MTU enforcement
    - nfp: fix infinite loop on umapping cleanup
    - tun: handle register_netdevice() failures properly
    - sctp: fully initialize the IPv6 address in sctp_v6_to_addr()
    - tipc: fix use-after-free
    - ipv6: reset fn->rr_ptr when replacing route
    - ipv6: repair fib6 tree in failure case
    - tcp: when rearming RTO, if RTO time is in past then fire RTO ASAP
    - net/mlx4_core: Enable 4K UAR if SRIOV module parameter is not enabled
    - irda: do not leak initialized list.dev to userspace
    - net: sched: fix NULL pointer dereference when action calls some targets
    - net_sched: fix order of queue length updates in qdisc_replace()
    - bpf, verifier: add additional patterns to evaluate_reg_imm_alu
    - bpf: fix mixed signed/unsigned derived min/max value bounds
    - bpf/verifier: fix min/max handling in BPF_SUB
    - Input: trackpoint - add new trackpoint firmware ID
    - Input: elan_i2c - add ELAN0602 ACPI ID to support Lenovo Yoga310
    - Input: ALPS - fix two-finger scroll breakage in right side on ALPS touchpad
    - KVM: s390: sthyi: fix sthyi inline assembly
    - KVM: s390: sthyi: fix specification exception detection
    - KVM: x86: simplify handling of PKRU
    - KVM, pkeys: do not use PKRU value in vcpu->arch.guest_fpu.state
    - KVM: x86: block guest protection keys unless the host has them enabled
    - ALSA: usb-audio: Add delay quirk for H650e/Jabra 550a USB headsets
    - ALSA: core: Fix unexpected error at replacing user TLV
    - ALSA: hda - Add stereo mic quirk for Lenovo G50-70 (17aa:3978)
    - ALSA: firewire: fix NULL pointer dereference when releasing uninitialized
      data of iso-resource
    - ALSA: firewire-motu: destroy stream data surely at failure of card
      initialization
    - ARCv2: SLC: Make sure busy bit is set properly for region ops
    - ARCv2: PAE40: Explicitly set MSB counterpart of SLC region ops addresses
    - ARCv2: PAE40: set MSB even if !CONFIG_ARC_HAS_...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.10.0-35.39

---------------
linux (4.10.0-35.39) zesty; urgency=low

  * linux: 4.10.0-35.39 -proposed tracker (LP: #1716606)

  * kernel panic -not syncing: Fatal exception: panic_on_oops (LP: #1708399)
    - SAUCE: s390/mm: fix local TLB flushing vs. detach of an mm address space
    - SAUCE: s390/mm: fix race on mm->context.flush_mm

  * CVE-2017-1000251
    - Bluetooth: Properly check L2CAP config option output buffer length

linux (4.10.0-34.38) zesty; urgency=low

  * linux: 4.10.0-34.38 -proposed tracker (LP: #1713470)

  * Ubuntu 16.04.03: perf tool does not count pm_run_inst_cmpl with rcode on
    POWER9 DD2.0 (LP: #1709964)
    - powerpc/perf: Fix Power9 test_adder fields

  * HID: multitouch: Support ALPS PTP Stick and Touchpad devices (LP: #1712481)
    - HID: multitouch: Support PTP Stick and Touchpad device
    - SAUCE: HID: multitouch: Support ALPS PTP stick with pid 0x120A

  * igb: Support using Broadcom 54616 as PHY (LP: #1712024)
    - SAUCE: igb: add support for using Broadcom 54616 as PHY

  * RPT related fixes missing in Ubuntu 16.04.3 (LP: #1709220)
    - powerpc/mm/radix: Optimise tlbiel flush all case
    - powerpc/mm/radix: Improve _tlbiel_pid to be usable for PWC flushes
    - powerpc/mm/radix: Improve TLB/PWC flushes
    - powerpc/mm/radix: Avoid flushing the PWC on every flush_tlb_range

  * AMD RV platforms with SNPS 3.1 USB controller stop responding (S3 issue)
    (LP: #1711098)
    - usb: xhci: Issue stop EP command only when the EP state is running

  * dma-buf: performance issue when looking up the fence status (LP: #1711096)
    - dma-buf: avoid scheduling on fence status query v2

  * IPR driver causes multipath to fail paths/stuck IO on Medium Errors
    (LP: #1682644)
    - scsi: ipr: do not set DID_PASSTHROUGH on CHECK CONDITION

  * Disable CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE (LP: #1709171)
    - [Config] CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=n for ppc64el

  * memory-hotplug test needs to be fixed (LP: #1710868)
    - selftests: typo correction for memory-hotplug test
    - selftests: check hot-pluggagble memory for memory-hotplug test
    - selftests: check percentage range for memory-hotplug test
    - selftests: add missing test name in memory-hotplug test
    - selftests: fix memory-hotplug test

  * Ubuntu 16.04.3: Qemu fails on P9 (LP: #1686019)
    - KVM: PPC: Pass kvm* to kvmppc_find_table()
    - KVM: PPC: Use preregistered memory API to access TCE list
    - KVM: PPC: VFIO: Add in-kernel acceleration for VFIO
    - powerpc/powernv/iommu: Add real mode version of iommu_table_ops::exchange()
    - powerpc/powernv/ioda2: Update iommu table base on ownership change
    - powerpc/iommu/vfio_spapr_tce: Cleanup iommu_table disposal
    - powerpc/vfio_spapr_tce: Add reference counting to iommu_table
    - powerpc/mmu: Add real mode support for IOMMU preregistered memory
    - KVM: PPC: Reserve KVM_CAP_SPAPR_TCE_VFIO capability number
    - KVM: PPC: Book3S HV: Add radix checks in real-mode hypercall handlers

  * [SRU][Zesty] [QDF2400] pl011 E44 erratum patch needed for 2.0 firmware and
    1.1 silicon (LP: #1709123)
    - tty: pl011: fix initialization or...