[18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part

Bug #1836860 reported by bugproxy
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Frank Heimes
linux (Ubuntu)
Fix Released
Undecided
Skipper Bug Screeners
Bionic
Fix Released
Medium
Unassigned
Disco
Fix Released
Medium
Unassigned

Bug Description

SRU Justification:
==================

[Impact]

* Enhanced CPU-MF hardware counters - fix lack of support of new hardware

[Fix]

* 46a984ffb86c8542fa510656fa8cb33befe8ee8f 46a984ff "s390/cpum_cf: Add support for CPU-MF SVN 6"

* 820bace734722715c643dcb5f74b502cb912d4eb 820bace "s390/cpumf: Add extended counter set definitions for model 8561 and 8562"

[Test Case]

* try CPU-Measurement Facility counter on z14 and next generation hardware - only IBM can do that now

[Regression Potential]

* The regression potential can be considered as low since these changes are limited to arch/s390

* do only touch perf_* code

* and mainly add code (rather than modify or remove)

[Other Info]

* first commit is taken from 5.2, second from 5.3-rc1
__________

Enhance the CPU-MF hardware counters for improved HW support . Kernel 5.2 commits and kernel 5.3 patch on top

Already requested for Ubuntu 19.10
kernel 5.2
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1834201
kernel 5.3 fix
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1836739

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-179262 severity-high targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Revision history for this message
Frank Heimes (fheimes) wrote :

If it should be added to bionic, it needs to be added to disco, too - to avoid any regressions during dist-upgrades.
So please provide a list of commit IDs (that apply cleanly to bionic master-next and disco master-next), respectively backports for a kernel SRU request.
Changing to Incomplete for now.

Changed in linux (Ubuntu):
status: New → Incomplete
Changed in ubuntu-z-systems:
status: New → Incomplete
importance: Undecided → High
assignee: nobody → Frank Heimes (frank-heimes)
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2019-07-17 07:42 EDT-------
This should be

Linux Kernel part
46a984ff s390/cpum_cf: Add support for CPU-MF SVN 6 (included in 5.2)
1c410fd s390/cpum_cf_diag: Add support for CPU-MF SVN 6 (included in 5.2)
820bace s390/cpumf: Add extended counter set definitions for model 8561 and 8562 (5.3-rc1)

s390-tools-part (all upstream available):
f110a77b cpumf: Add support for CPU-Measurement Facility counters SVN 6

libpfm4 part (all upsteam available):
b831b04 s390/cpumf: add support for machine type 8561
c08003f s390/cpumf: add support for counter second version number 6
dc643f4 s390/cpumf: add IBM z14 ZR1 support

Revision history for this message
Frank Heimes (fheimes) wrote :

Just for reference: The following s390-tools enhancement:
s390-tools-part (all upstream available)
f110a77b cpumf: Add support for CPU-Measurement Facility counters SVN 6
is already addressed in LP 1836862.

But libpfm4 is not yet addressed.

Revision history for this message
Frank Heimes (fheimes) wrote :

Just for completeness reasons:
The request to update/patch libpfm4 came in now with LP 1836532.

Frank Heimes (fheimes)
description: updated
Revision history for this message
Frank Heimes (fheimes) wrote :

I tried to cherry-pick the commits 46a984ff, 1c410fd and 820bace from linux master into bionic master-next, but they don't apply cleanly and git complains about a missing file "arch/s390/kernel/perf_cpum_cf_diag.c".
I can 'git add' it, but thought I better look in the git log for further commits needed and found:
b6ffdf2 s390/cpumf: Fix warning from check_processor_id
fe5908b s390/cpum_cf_diag: Add support for s390 counter facility diagnostic trace
But while trying to cherry-pick these on top, next file is missing "arch/s390/include/asm/cpu_mcf.h".

Before I further go down the vicious circle, please (@IBM) can you come up either with a complete list of commit IDs that allow to cherry-pick cleanly based on bionic master-next (and ideally also disco master -next) or with a backport/PR for 4.15?
Thx

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-24 07:23 EDT-------
I have issued the following commands on my ubuntu 18.04 system:

1. git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic
2. cd bionic
3. git checkout master-next
4. root@s8360046:~/krn1802git/bionic# git branch
* master-next

I then applied the following patches:
- 46a984ffb86 [PATCH] s390/cpum_cf: Add support for CPU-MF SVN 6
- 820bace7347 [PATCH] s390/cpumf: Add extended counter set definitions for model 8561 and 8562

Both patches applied with some noise about line mismatch line numbers, such as:

root@s8360046:~/krn1802git/bionic# patch -p1 < ../ubuntu-krn1802/0001-s390-cpum_cf-Add-support-for-CPU-MF-SVN-6.patch
patching file arch/s390/kernel/perf_cpum_cf.c
Hunk #2 succeeded at 88 (offset 62 lines).
Hunk #3 succeeded at 112 (offset 62 lines).
patching file arch/s390/kernel/perf_cpum_cf_events.c
Hunk #1 succeeded at 30 (offset -1 lines).
Hunk #2 succeeded at 265 (offset -1 lines).
Hunk #3 succeeded at 589 (offset -1 lines).

It turns out that you do do not need this patch:
- 1c410fd6a56 [PATCH] s390/cpum_cf_diag: Add support for CPU-MF SVN 6
It requires a predecessor patch which introduced a new CPU Measurement
facility counter device driver for s390 with kernel 5.1:

- fe5908bccc56 s390/cpum_cf_diag: Add support for s390 counter facility diagnostic trace

We do not need this to be backported to 4.15. So omit commit 1c410fd6a56.

I can sent you the remaining 2 patches reworked to apply cleanly without noise,
however with a different commit id. Let me know if you want them.
~

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-24 08:55 EDT-------
I have done the same for disco repository:

1. # git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/disco
2. # cd disco/
3. # git checkout master-next

Now apply both patches (with some noise about line mismatches)

- 46a984ffb86 [PATCH] s390/cpum_cf: Add support for CPU-MF SVN 6
- 820bace7347 [PATCH] s390/cpumf: Add extended counter set definitions for model 8561 and 8562

You can not apply these patches
- fe5908bccc56 s390/cpum_cf_diag: Add support for s390 counter facility diagnostic trace
- 1c410fd6a56 [PATCH] s390/cpum_cf_diag: Add support for CPU-MF SVN 6

The first one needs a patch series which re-organizes some header files and other internal stuff
for the CPU Measurement Facility diagnostic counter set which we can defer until Linux 5.3.

I do not want to backport this to 5.0.

Revision history for this message
Frank Heimes (fheimes) wrote :

Thanks for checking - but no need to sent the patches.
Commit 1c410fd6a56 was the 'troublemaker' - without that I can cherry-pick and cleanly apply the remaining two for bionic and disco.

Frank Heimes (fheimes)
description: updated
description: updated
Revision history for this message
Frank Heimes (fheimes) wrote :
Changed in linux (Ubuntu):
status: Incomplete → In Progress
Changed in ubuntu-z-systems:
status: Incomplete → In Progress
information type: Private → Public
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (37.9 KiB)

This bug was fixed in the package linux - 5.2.0-10.11

---------------
linux (5.2.0-10.11) eoan; urgency=medium

  * eoan/linux: 5.2.0-10.11 -proposed tracker (LP: #1838113)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

  * Eoan update: v5.2.4 upstream stable release (LP: #1838428)
    - bnx2x: Prevent load reordering in tx completion processing
    - caif-hsi: fix possible deadlock in cfhsi_exit_module()
    - hv_netvsc: Fix extra rcu_read_unlock in netvsc_recv_callback()
    - igmp: fix memory leak in igmpv3_del_delrec()
    - ipv4: don't set IPv6 only flags to IPv4 addresses
    - ipv6: rt6_check should return NULL if 'from' is NULL
    - ipv6: Unlink sibling route in case of failure
    - net: bcmgenet: use promisc for unsupported filters
    - net: dsa: mv88e6xxx: wait after reset deactivation
    - net: make skb_dst_force return true when dst is refcounted
    - net: neigh: fix multiple neigh timer scheduling
    - net: openvswitch: fix csum updates for MPLS actions
    - net: phy: sfp: hwmon: Fix scaling of RX power
    - net_sched: unset TCQ_F_CAN_BYPASS when adding filters
    - net: stmmac: Re-work the queue selection for TSO packets
    - net/tls: make sure offload also gets the keys wiped
    - nfc: fix potential illegal memory access
    - r8169: fix issue with confused RX unit after PHY power-down on RTL8411b
    - rxrpc: Fix send on a connected, but unbound socket
    - sctp: fix error handling on stream scheduler initialization
    - sctp: not bind the socket in sctp_connect
    - sky2: Disable MSI on ASUS P6T
    - tcp: be more careful in tcp_fragment()
    - tcp: fix tcp_set_congestion_control() use from bpf hook
    - tcp: Reset bytes_acked and bytes_received when disconnecting
    - vrf: make sure skb->data contains ip header to make routing
    - net/mlx5e: IPoIB, Add error path in mlx5_rdma_setup_rn
    - net: bridge: mcast: fix stale nsrcs pointer in igmp3/mld2 report handling
    - net: bridge: mcast: fix stale ipv6 hdr pointer when handling v6 query
    - net: bridge: don't cache ether dest pointer on input
    - net: bridge: stp: don't cache eth dest pointer before skb pull
    - macsec: fix use-after-free of skb during RX
    - macsec: fix checksumming after decryption
    - netrom: fix a memory leak in nr_rx_frame()
    - netrom: hold sock when setting skb->destructor
    - selftests: txring_overwrite: fix incorrect test of mmap() return value
    - net/tls: fix poll ignoring partially copied records
    - net/tls: reject offload of TLS 1.3
    - net/mlx5e: Fix port tunnel GRE entropy control
    - net/mlx5e: Rx, Fix checksum calculation for new hardware
    - net/mlx5e: Fix return value from timeout recover function
    - net/mlx5e: Fix error flow in tx reporter diagnose
    - bnxt_en: Fix VNIC accounting when enabling aRFS on 57500 chips.
    - mlxsw: spectrum_dcb: Configure DSCP map as the last rule is removed
    - net/mlx5: E-Switch, Fix default encap mode
    - mlxsw: spectrum: Do not process learned records with a dummy FID
    - dma-buf: balance refcount inbalance
    - dma-buf: Discard old fence_excl on retrying get_fences_rcu for realloc
    - Revert "gpio/spi: Fix spi-gpio...

Changed in linux (Ubuntu):
status: In Progress → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Stefan Bader (smb)
Changed in linux (Ubuntu Bionic):
importance: Undecided → Medium
status: New → Confirmed
Stefan Bader (smb)
Changed in linux (Ubuntu Disco):
importance: Undecided → Medium
status: New → Confirmed
Changed in linux (Ubuntu Bionic):
status: Confirmed → Fix Committed
Changed in linux (Ubuntu Disco):
status: Confirmed → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-08-15 07:36 EDT-------
Function already verified upstream by IBM

Frank Heimes (fheimes)
tags: added: verification-done-disco
removed: verification-needed-disco
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (34.4 KiB)

This bug was fixed in the package linux - 5.0.0-27.28

---------------
linux (5.0.0-27.28) disco; urgency=medium

  * disco/linux: 5.0.0-27.28 -proposed tracker (LP: #1840816)

  * [Potential Regression] System crashes when running ftrace test in
    ubuntu_kernel_selftests (LP: #1840750)
    - x86/kprobes: Set instruction page as executable

linux (5.0.0-26.27) disco; urgency=medium

  * disco/linux: 5.0.0-26.27 -proposed tracker (LP: #1839972)

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts

  * alsa/hdmi: add icelake hdmi audio support for a Dell machine (LP: #1836916)
    - ALSA: hda: hdmi - add Icelake support
    - ALSA: hda/hdmi - Remove duplicated define
    - ALSA: hda/hdmi - Fix i915 reverse port/pin mapping

  * input/mouse: alps trackpoint-only device doesn't work (LP: #1836752)
    - Input: alps - don't handle ALPS cs19 trackpoint-only device
    - Input: alps - fix a mismatch between a condition check and its comment

  * [18.04 FEAT] Enhanced hardware support (LP: #1836857)
    - s390: report new CPU capabilities
    - s390: add alignment hints to vector load and store

  * System does not auto detect disconnection of external monitor (LP: #1835001)
    - drm/i915: Add support for retrying hotplug
    - drm/i915: Enable hotplug retry

  * [18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part (LP: #1836860)
    - s390/cpum_cf: Add support for CPU-MF SVN 6
    - s390/cpumf: Add extended counter set definitions for model 8561 and 8562

  * EeePC 1005px laptop backlight is off after system boot up (LP: #1837117)
    - platform/x86: asus-wmi: Only Tell EC the OS will handle display hotkeys from
      asus_nb_wmi

  * br_netfilter: namespace sysctl operations (LP: #1836910)
    - netfilter: bridge: port sysctls to use brnf_net
    - netfilter: bridge: namespace bridge netfilter sysctls
    - netfilter: bridge: prevent UAF in brnf_exit_net()

  * ideapad_laptop disables WiFi/BT radios on Lenovo Y540 (LP: #1837136)
    - platform/x86: ideapad-laptop: Remove no_hw_rfkill_list

  * shiftfs: allow overlayfs (LP: #1838677)
    - SAUCE: shiftfs: enable overlayfs on shiftfs

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665)
    - bcache: never writeback a discard operation
    - bcache: improve bcache_reboot()
    - SAUCE: bcache: fix deadlock in bcache_allocator

  * Regressions in CMA allocation rework (LP: #1839395)
    - dma-contiguous: do not overwrite align in dma_alloc_contiguous()
    - dma-contiguous: page-align the size in dma_free_contiguous()

  * CVE-2019-3900
    - vhost: introduce vhost_exceeds_weight()
    - vhost_net: fix possible infinite loop
    - vhost: vsock: add weight support
    - vhost: scsi: add weight support

  * Disco update: 5.0.21 upstream stable release (LP: #1837518)
    - bonding/802.3ad: fix slave link initialization transition states
    - cxgb4: offload VLAN flows regardless of VLAN ethtype
    - inet: switch IP ID generator to siphash
    - ipv4/igmp: fix another memory leak in igmpv3_del_delrec()
    - ipv4/igmp: fix build error if !CONFIG_IP_MULTICAST
    - ipv6: Consider sk_bound_dev_if when binding a raw socket to an address
    - ipv6: Fix redi...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (235.3 KiB)

This bug was fixed in the package linux - 4.15.0-60.67

---------------
linux (4.15.0-60.67) bionic; urgency=medium

  * bionic/linux: 4.15.0-60.67 -proposed tracker (LP: #1841086)

  * [Regression] net test from ubuntu_kernel_selftests failed due to bpf test
    compilation issue (LP: #1840935)
    - SAUCE: Fix "bpf: relax verifier restriction on BPF_MOV | BPF_ALU"

  * [Regression] failed to compile seccomp test from ubuntu_kernel_selftests
    (LP: #1840932)
    - Revert "selftests: skip seccomp get_metadata test if not real root"

  * Packaging resync (LP: #1786013)
    - [Packaging] resync getabis

linux (4.15.0-59.66) bionic; urgency=medium

  * bionic/linux: 4.15.0-59.66 -proposed tracker (LP: #1840006)

  * zfs not completely removed from bionic tree (LP: #1840051)
    - SAUCE: (noup) remove completely the zfs code

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts

  * [18.04 FEAT] Enhanced hardware support (LP: #1836857)
    - s390: report new CPU capabilities
    - s390: add alignment hints to vector load and store

  * [18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part (LP: #1836860)
    - s390/cpum_cf: Add support for CPU-MF SVN 6
    - s390/cpumf: Add extended counter set definitions for model 8561 and 8562

  * ideapad_laptop disables WiFi/BT radios on Lenovo Y540 (LP: #1837136)
    - platform/x86: ideapad-laptop: Remove no_hw_rfkill_list

  * Stacked onexec transitions fail when under NO NEW PRIVS restrictions
    (LP: #1839037)
    - SAUCE: apparmor: fix nnp subset check failure when, stacking

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665) // Tight
    timeout for bcache removal causes spurious failures (LP: #1796292)
    - SAUCE: bcache: fix deadlock in bcache_allocator

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665)
    - bcache: never writeback a discard operation
    - bcache: improve bcache_reboot()
    - bcache: fix writeback target calc on large devices
    - bcache: add journal statistic
    - bcache: fix high CPU occupancy during journal
    - bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set
    - bcache: fix incorrect sysfs output value of strip size
    - bcache: fix error return value in memory shrink
    - bcache: fix using of loop variable in memory shrink
    - bcache: Fix indentation
    - bcache: Add __printf annotation to __bch_check_keys()
    - bcache: Annotate switch fall-through
    - bcache: Fix kernel-doc warnings
    - bcache: Remove an unused variable
    - bcache: Suppress more warnings about set-but-not-used variables
    - bcache: Reduce the number of sparse complaints about lock imbalances
    - bcache: Fix a compiler warning in bcache_device_init()
    - bcache: Move couple of string arrays to sysfs.c
    - bcache: Move couple of functions to sysfs.c
    - bcache: Replace bch_read_string_list() by __sysfs_match_string()

  * linux hwe i386 kernel 5.0.0-21.22~18.04.1 crashes on Lenovo x220
    (LP: #1838115)
    - x86/mm: Check for pfn instead of page in vmalloc_sync_one()
    - x86/mm: Sync also unmappings in vmalloc_sync_all()
    - mm/vmalloc.c: add priority threshold to __purge_vmap_area_lazy()...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-09-04 03:51 EDT-------
IBM Bugzilla status -> closed, Fix Released with Bionic

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.