[18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part

Bug #1836860 reported by bugproxy on 2019-07-17
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
High
Frank Heimes
linux (Ubuntu)
Undecided
Skipper Bug Screeners
Bionic
Medium
Unassigned
Disco
Medium
Unassigned

Bug Description

SRU Justification:
==================

[Impact]

* Enhanced CPU-MF hardware counters - fix lack of support of new hardware

[Fix]

* 46a984ffb86c8542fa510656fa8cb33befe8ee8f 46a984ff "s390/cpum_cf: Add support for CPU-MF SVN 6"

* 820bace734722715c643dcb5f74b502cb912d4eb 820bace "s390/cpumf: Add extended counter set definitions for model 8561 and 8562"

[Test Case]

* try CPU-Measurement Facility counter on z14 and next generation hardware - only IBM can do that now

[Regression Potential]

* The regression potential can be considered as low since these changes are limited to arch/s390

* do only touch perf_* code

* and mainly add code (rather than modify or remove)

[Other Info]

* first commit is taken from 5.2, second from 5.3-rc1
__________

Enhance the CPU-MF hardware counters for improved HW support . Kernel 5.2 commits and kernel 5.3 patch on top

Already requested for Ubuntu 19.10
kernel 5.2
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1834201
kernel 5.3 fix
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1836739

bugproxy (bugproxy) on 2019-07-17
tags: added: architecture-s39064 bugnameltc-179262 severity-high targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (fheimes) wrote :

If it should be added to bionic, it needs to be added to disco, too - to avoid any regressions during dist-upgrades.
So please provide a list of commit IDs (that apply cleanly to bionic master-next and disco master-next), respectively backports for a kernel SRU request.
Changing to Incomplete for now.

Changed in linux (Ubuntu):
status: New → Incomplete
Changed in ubuntu-z-systems:
status: New → Incomplete
importance: Undecided → High
assignee: nobody → Frank Heimes (frank-heimes)

------- Comment From <email address hidden> 2019-07-17 07:42 EDT-------
This should be

Linux Kernel part
46a984ff s390/cpum_cf: Add support for CPU-MF SVN 6 (included in 5.2)
1c410fd s390/cpum_cf_diag: Add support for CPU-MF SVN 6 (included in 5.2)
820bace s390/cpumf: Add extended counter set definitions for model 8561 and 8562 (5.3-rc1)

s390-tools-part (all upstream available):
f110a77b cpumf: Add support for CPU-Measurement Facility counters SVN 6

libpfm4 part (all upsteam available):
b831b04 s390/cpumf: add support for machine type 8561
c08003f s390/cpumf: add support for counter second version number 6
dc643f4 s390/cpumf: add IBM z14 ZR1 support

Frank Heimes (fheimes) wrote :

Just for reference: The following s390-tools enhancement:
s390-tools-part (all upstream available)
f110a77b cpumf: Add support for CPU-Measurement Facility counters SVN 6
is already addressed in LP 1836862.

But libpfm4 is not yet addressed.

Frank Heimes (fheimes) wrote :

Just for completeness reasons:
The request to update/patch libpfm4 came in now with LP 1836532.

Frank Heimes (fheimes) on 2019-07-18
description: updated
Frank Heimes (fheimes) wrote :

I tried to cherry-pick the commits 46a984ff, 1c410fd and 820bace from linux master into bionic master-next, but they don't apply cleanly and git complains about a missing file "arch/s390/kernel/perf_cpum_cf_diag.c".
I can 'git add' it, but thought I better look in the git log for further commits needed and found:
b6ffdf2 s390/cpumf: Fix warning from check_processor_id
fe5908b s390/cpum_cf_diag: Add support for s390 counter facility diagnostic trace
But while trying to cherry-pick these on top, next file is missing "arch/s390/include/asm/cpu_mcf.h".

Before I further go down the vicious circle, please (@IBM) can you come up either with a complete list of commit IDs that allow to cherry-pick cleanly based on bionic master-next (and ideally also disco master -next) or with a backport/PR for 4.15?
Thx

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-24 07:23 EDT-------
I have issued the following commands on my ubuntu 18.04 system:

1. git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic
2. cd bionic
3. git checkout master-next
4. root@s8360046:~/krn1802git/bionic# git branch
* master-next

I then applied the following patches:
- 46a984ffb86 [PATCH] s390/cpum_cf: Add support for CPU-MF SVN 6
- 820bace7347 [PATCH] s390/cpumf: Add extended counter set definitions for model 8561 and 8562

Both patches applied with some noise about line mismatch line numbers, such as:

root@s8360046:~/krn1802git/bionic# patch -p1 < ../ubuntu-krn1802/0001-s390-cpum_cf-Add-support-for-CPU-MF-SVN-6.patch
patching file arch/s390/kernel/perf_cpum_cf.c
Hunk #2 succeeded at 88 (offset 62 lines).
Hunk #3 succeeded at 112 (offset 62 lines).
patching file arch/s390/kernel/perf_cpum_cf_events.c
Hunk #1 succeeded at 30 (offset -1 lines).
Hunk #2 succeeded at 265 (offset -1 lines).
Hunk #3 succeeded at 589 (offset -1 lines).

It turns out that you do do not need this patch:
- 1c410fd6a56 [PATCH] s390/cpum_cf_diag: Add support for CPU-MF SVN 6
It requires a predecessor patch which introduced a new CPU Measurement
facility counter device driver for s390 with kernel 5.1:

- fe5908bccc56 s390/cpum_cf_diag: Add support for s390 counter facility diagnostic trace

We do not need this to be backported to 4.15. So omit commit 1c410fd6a56.

I can sent you the remaining 2 patches reworked to apply cleanly without noise,
however with a different commit id. Let me know if you want them.
~

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-24 08:55 EDT-------
I have done the same for disco repository:

1. # git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/disco
2. # cd disco/
3. # git checkout master-next

Now apply both patches (with some noise about line mismatches)

- 46a984ffb86 [PATCH] s390/cpum_cf: Add support for CPU-MF SVN 6
- 820bace7347 [PATCH] s390/cpumf: Add extended counter set definitions for model 8561 and 8562

You can not apply these patches
- fe5908bccc56 s390/cpum_cf_diag: Add support for s390 counter facility diagnostic trace
- 1c410fd6a56 [PATCH] s390/cpum_cf_diag: Add support for CPU-MF SVN 6

The first one needs a patch series which re-organizes some header files and other internal stuff
for the CPU Measurement Facility diagnostic counter set which we can defer until Linux 5.3.

I do not want to backport this to 5.0.

Frank Heimes (fheimes) wrote :

Thanks for checking - but no need to sent the patches.
Commit 1c410fd6a56 was the 'troublemaker' - without that I can cherry-pick and cleanly apply the remaining two for bionic and disco.

Frank Heimes (fheimes) on 2019-07-24
description: updated
description: updated
Frank Heimes (fheimes) wrote :
Changed in linux (Ubuntu):
status: Incomplete → In Progress
Changed in ubuntu-z-systems:
status: Incomplete → In Progress
information type: Private → Public
Launchpad Janitor (janitor) wrote :
Download full text (37.9 KiB)

This bug was fixed in the package linux - 5.2.0-10.11

---------------
linux (5.2.0-10.11) eoan; urgency=medium

  * eoan/linux: 5.2.0-10.11 -proposed tracker (LP: #1838113)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

  * Eoan update: v5.2.4 upstream stable release (LP: #1838428)
    - bnx2x: Prevent load reordering in tx completion processing
    - caif-hsi: fix possible deadlock in cfhsi_exit_module()
    - hv_netvsc: Fix extra rcu_read_unlock in netvsc_recv_callback()
    - igmp: fix memory leak in igmpv3_del_delrec()
    - ipv4: don't set IPv6 only flags to IPv4 addresses
    - ipv6: rt6_check should return NULL if 'from' is NULL
    - ipv6: Unlink sibling route in case of failure
    - net: bcmgenet: use promisc for unsupported filters
    - net: dsa: mv88e6xxx: wait after reset deactivation
    - net: make skb_dst_force return true when dst is refcounted
    - net: neigh: fix multiple neigh timer scheduling
    - net: openvswitch: fix csum updates for MPLS actions
    - net: phy: sfp: hwmon: Fix scaling of RX power
    - net_sched: unset TCQ_F_CAN_BYPASS when adding filters
    - net: stmmac: Re-work the queue selection for TSO packets
    - net/tls: make sure offload also gets the keys wiped
    - nfc: fix potential illegal memory access
    - r8169: fix issue with confused RX unit after PHY power-down on RTL8411b
    - rxrpc: Fix send on a connected, but unbound socket
    - sctp: fix error handling on stream scheduler initialization
    - sctp: not bind the socket in sctp_connect
    - sky2: Disable MSI on ASUS P6T
    - tcp: be more careful in tcp_fragment()
    - tcp: fix tcp_set_congestion_control() use from bpf hook
    - tcp: Reset bytes_acked and bytes_received when disconnecting
    - vrf: make sure skb->data contains ip header to make routing
    - net/mlx5e: IPoIB, Add error path in mlx5_rdma_setup_rn
    - net: bridge: mcast: fix stale nsrcs pointer in igmp3/mld2 report handling
    - net: bridge: mcast: fix stale ipv6 hdr pointer when handling v6 query
    - net: bridge: don't cache ether dest pointer on input
    - net: bridge: stp: don't cache eth dest pointer before skb pull
    - macsec: fix use-after-free of skb during RX
    - macsec: fix checksumming after decryption
    - netrom: fix a memory leak in nr_rx_frame()
    - netrom: hold sock when setting skb->destructor
    - selftests: txring_overwrite: fix incorrect test of mmap() return value
    - net/tls: fix poll ignoring partially copied records
    - net/tls: reject offload of TLS 1.3
    - net/mlx5e: Fix port tunnel GRE entropy control
    - net/mlx5e: Rx, Fix checksum calculation for new hardware
    - net/mlx5e: Fix return value from timeout recover function
    - net/mlx5e: Fix error flow in tx reporter diagnose
    - bnxt_en: Fix VNIC accounting when enabling aRFS on 57500 chips.
    - mlxsw: spectrum_dcb: Configure DSCP map as the last rule is removed
    - net/mlx5: E-Switch, Fix default encap mode
    - mlxsw: spectrum: Do not process learned records with a dummy FID
    - dma-buf: balance refcount inbalance
    - dma-buf: Discard old fence_excl on retrying get_fences_rcu for realloc
    - Revert "gpio/spi: Fix spi-gpio...

Changed in linux (Ubuntu):
status: In Progress → Fix Released
Frank Heimes (fheimes) on 2019-08-09
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Stefan Bader (smb) on 2019-08-12
Changed in linux (Ubuntu Bionic):
importance: Undecided → Medium
status: New → Confirmed
Stefan Bader (smb) on 2019-08-12
Changed in linux (Ubuntu Disco):
importance: Undecided → Medium
status: New → Confirmed
Changed in linux (Ubuntu Bionic):
status: Confirmed → Fix Committed
Changed in linux (Ubuntu Disco):
status: Confirmed → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-08-15 07:36 EDT-------
Function already verified upstream by IBM

Frank Heimes (fheimes) on 2019-08-15
tags: added: verification-done-disco
removed: verification-needed-disco

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial
Launchpad Janitor (janitor) wrote :
Download full text (34.4 KiB)

This bug was fixed in the package linux - 5.0.0-27.28

---------------
linux (5.0.0-27.28) disco; urgency=medium

  * disco/linux: 5.0.0-27.28 -proposed tracker (LP: #1840816)

  * [Potential Regression] System crashes when running ftrace test in
    ubuntu_kernel_selftests (LP: #1840750)
    - x86/kprobes: Set instruction page as executable

linux (5.0.0-26.27) disco; urgency=medium

  * disco/linux: 5.0.0-26.27 -proposed tracker (LP: #1839972)

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts

  * alsa/hdmi: add icelake hdmi audio support for a Dell machine (LP: #1836916)
    - ALSA: hda: hdmi - add Icelake support
    - ALSA: hda/hdmi - Remove duplicated define
    - ALSA: hda/hdmi - Fix i915 reverse port/pin mapping

  * input/mouse: alps trackpoint-only device doesn't work (LP: #1836752)
    - Input: alps - don't handle ALPS cs19 trackpoint-only device
    - Input: alps - fix a mismatch between a condition check and its comment

  * [18.04 FEAT] Enhanced hardware support (LP: #1836857)
    - s390: report new CPU capabilities
    - s390: add alignment hints to vector load and store

  * System does not auto detect disconnection of external monitor (LP: #1835001)
    - drm/i915: Add support for retrying hotplug
    - drm/i915: Enable hotplug retry

  * [18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part (LP: #1836860)
    - s390/cpum_cf: Add support for CPU-MF SVN 6
    - s390/cpumf: Add extended counter set definitions for model 8561 and 8562

  * EeePC 1005px laptop backlight is off after system boot up (LP: #1837117)
    - platform/x86: asus-wmi: Only Tell EC the OS will handle display hotkeys from
      asus_nb_wmi

  * br_netfilter: namespace sysctl operations (LP: #1836910)
    - netfilter: bridge: port sysctls to use brnf_net
    - netfilter: bridge: namespace bridge netfilter sysctls
    - netfilter: bridge: prevent UAF in brnf_exit_net()

  * ideapad_laptop disables WiFi/BT radios on Lenovo Y540 (LP: #1837136)
    - platform/x86: ideapad-laptop: Remove no_hw_rfkill_list

  * shiftfs: allow overlayfs (LP: #1838677)
    - SAUCE: shiftfs: enable overlayfs on shiftfs

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665)
    - bcache: never writeback a discard operation
    - bcache: improve bcache_reboot()
    - SAUCE: bcache: fix deadlock in bcache_allocator

  * Regressions in CMA allocation rework (LP: #1839395)
    - dma-contiguous: do not overwrite align in dma_alloc_contiguous()
    - dma-contiguous: page-align the size in dma_free_contiguous()

  * CVE-2019-3900
    - vhost: introduce vhost_exceeds_weight()
    - vhost_net: fix possible infinite loop
    - vhost: vsock: add weight support
    - vhost: scsi: add weight support

  * Disco update: 5.0.21 upstream stable release (LP: #1837518)
    - bonding/802.3ad: fix slave link initialization transition states
    - cxgb4: offload VLAN flows regardless of VLAN ethtype
    - inet: switch IP ID generator to siphash
    - ipv4/igmp: fix another memory leak in igmpv3_del_delrec()
    - ipv4/igmp: fix build error if !CONFIG_IP_MULTICAST
    - ipv6: Consider sk_bound_dev_if when binding a raw socket to an address
    - ipv6: Fix redi...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (235.3 KiB)

This bug was fixed in the package linux - 4.15.0-60.67

---------------
linux (4.15.0-60.67) bionic; urgency=medium

  * bionic/linux: 4.15.0-60.67 -proposed tracker (LP: #1841086)

  * [Regression] net test from ubuntu_kernel_selftests failed due to bpf test
    compilation issue (LP: #1840935)
    - SAUCE: Fix "bpf: relax verifier restriction on BPF_MOV | BPF_ALU"

  * [Regression] failed to compile seccomp test from ubuntu_kernel_selftests
    (LP: #1840932)
    - Revert "selftests: skip seccomp get_metadata test if not real root"

  * Packaging resync (LP: #1786013)
    - [Packaging] resync getabis

linux (4.15.0-59.66) bionic; urgency=medium

  * bionic/linux: 4.15.0-59.66 -proposed tracker (LP: #1840006)

  * zfs not completely removed from bionic tree (LP: #1840051)
    - SAUCE: (noup) remove completely the zfs code

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts

  * [18.04 FEAT] Enhanced hardware support (LP: #1836857)
    - s390: report new CPU capabilities
    - s390: add alignment hints to vector load and store

  * [18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part (LP: #1836860)
    - s390/cpum_cf: Add support for CPU-MF SVN 6
    - s390/cpumf: Add extended counter set definitions for model 8561 and 8562

  * ideapad_laptop disables WiFi/BT radios on Lenovo Y540 (LP: #1837136)
    - platform/x86: ideapad-laptop: Remove no_hw_rfkill_list

  * Stacked onexec transitions fail when under NO NEW PRIVS restrictions
    (LP: #1839037)
    - SAUCE: apparmor: fix nnp subset check failure when, stacking

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665) // Tight
    timeout for bcache removal causes spurious failures (LP: #1796292)
    - SAUCE: bcache: fix deadlock in bcache_allocator

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665)
    - bcache: never writeback a discard operation
    - bcache: improve bcache_reboot()
    - bcache: fix writeback target calc on large devices
    - bcache: add journal statistic
    - bcache: fix high CPU occupancy during journal
    - bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set
    - bcache: fix incorrect sysfs output value of strip size
    - bcache: fix error return value in memory shrink
    - bcache: fix using of loop variable in memory shrink
    - bcache: Fix indentation
    - bcache: Add __printf annotation to __bch_check_keys()
    - bcache: Annotate switch fall-through
    - bcache: Fix kernel-doc warnings
    - bcache: Remove an unused variable
    - bcache: Suppress more warnings about set-but-not-used variables
    - bcache: Reduce the number of sparse complaints about lock imbalances
    - bcache: Fix a compiler warning in bcache_device_init()
    - bcache: Move couple of string arrays to sysfs.c
    - bcache: Move couple of functions to sysfs.c
    - bcache: Replace bch_read_string_list() by __sysfs_match_string()

  * linux hwe i386 kernel 5.0.0-21.22~18.04.1 crashes on Lenovo x220
    (LP: #1838115)
    - x86/mm: Check for pfn instead of page in vmalloc_sync_one()
    - x86/mm: Sync also unmappings in vmalloc_sync_all()
    - mm/vmalloc.c: add priority threshold to __purge_vmap_area_lazy()...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-09-04 03:51 EDT-------
IBM Bugzilla status -> closed, Fix Released with Bionic

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers