[UBUNTU 20.04] Cannot run zcryptstats in the Hosting Appliance

Bug #1903984 reported by bugproxy on 2020-11-12
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Medium
Skipper Bug Screeners
s390-tools (Ubuntu)
Status tracked in Hirsute
Focal
Medium
Frank Heimes
Groovy
Medium
Frank Heimes
Hirsute
Medium
Unassigned

Bug Description

SRU Justification:
==================

[Impact]

 * If multiple s390x crypto domains are available,
   the output of all domains does no longer fit into the output buffer of one CHSC call.

 * Thus, the response contains a partial-response-snippet that indicates that there is more data to be retrieved.

 * The remaining data is then retrieved in a subsequent CHSC call,
   but there is an error in zcryptstats in populating the request with the next domain number to obtain data for.

 * This results in the fact that it retrieves the data for domains 1 to n over and over again,
   and always gets the partial-response snippet, hence this loops indefinitely.

[Test Case]

 * An Ubuntu Server 20.04 system (or newer) is needed on z/VM or LPAR with two or more crypto domains accessible.

 * Now call 'zcryptstats --all -V' and verify if the command completes successfully (within seconds) or hangs (loops).

[Where problems could occur]

 * The patch fixes the code that passes the correct next-domain to the subsequent CHSC call of a partial response.

 * In case done wrong even no data at all from any of the domains is retrieved,

 * or the tool may even try to retrieve data from a domain that has an invalid index, which will lead to an error (the rc is captured).

 * But the code changes are marginal and pretty straight forward, and the cases where scdmd_area.request.first_drid is handled are now enclosed by proper 'if/else' statement.

 * This patch was already upstream accepted and a patched s390-tools package was build and shared in a PPA for further testing.
__________
---Problem Description---
I have had trouble collecting data via the zcryptstats (https://github.com/ibm-s390-tools/s390-tools/blob/master/zconf/zcrypt/zcryptstats.c) tool for our Hosting Appliance - VPC environment. Basically, the tool reports data for only the last two domain IDs. Further, it hangs if I collect data for a complete crypto card, and then I have to kill the ssh session to get back.

strace attached, ltrace not available and can't be installed due to no Internet access.

===================

Addl. Information:

This seems to be a problem with the handling of partial responses of the SCDMD CHSC command.

When many domains are available, then the output of all domains does not fit into the output buffer of one CHSC call. Thus, the response contains a partial-response-bit that indicates that there is more data to be retrieved. This is done in a subsequent CHSC call, but there is an error in zcryptstats with populating the request with the next domain number to obtain data for. This results in the fact that it retrieves the data for domains 1 to n over and over again, and always gets the partial-response bit set. So this loops forever.

===================
The fix for this in now upstream:
https://github.com/ibm-s390-tools/s390-tools/commit/cf2311f1f1de17435b49ba8c8697be91705ba031

This need to be integrated in all Ubuntu distros, from 20.04 on. F/G/H

Related branches

bugproxy (bugproxy) on 2020-11-12
tags: added: architecture-s39064 bugnameltc-189183 severity-medium targetmilestone-inin2004
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (fheimes) on 2020-11-12
affects: linux (Ubuntu) → s390-tools
Changed in ubuntu-z-systems:
importance: Undecided → Medium
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
Frank Heimes (fheimes) on 2020-11-12
Changed in s390-tools:
status: New → Invalid
summary: - Cannot run zcryptstats in the Hosting Appliance
+ [UBUNTU 20.04] Cannot run zcryptstats in the Hosting Appliance
Changed in s390-tools (Ubuntu Hirsute):
status: New → Confirmed

------- Comment From <email address hidden> 2021-01-18 05:57 EDT-------
@Canonical, please provide an update for this update request. Many thx in advance

Frank Heimes (fheimes) wrote :

A patched s390-tools package was build for hirsute and shared in the following PPA for further tesing: https://launchpad.net/~fheimes/+archive/ubuntu/lp1903984

description: updated
Frank Heimes (fheimes) on 2021-01-19
Changed in ubuntu-z-systems:
status: New → Triaged
Changed in s390-tools (Ubuntu Focal):
status: New → Triaged
Changed in s390-tools (Ubuntu Groovy):
status: New → Triaged
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2021-01-19 04:55 EDT-------
The git commit should apply to all requested levels. If tested is also required for non-hirsuite levels, please provide a PPA, at least for focal.. thx

Frank Heimes (fheimes) wrote :

I can confirm that the patch applies fine to H, G and F.
The PPA contains packages for all 3 Ubuntu releases.

The attachment "debdiff_hirsute.diff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Frank Heimes (fheimes) on 2021-01-19
Changed in s390-tools (Ubuntu Focal):
assignee: nobody → Frank Heimes (fheimes)
Changed in s390-tools (Ubuntu Groovy):
assignee: nobody → Frank Heimes (fheimes)
Frank Heimes (fheimes) wrote :

Default Comment by Bridge

Frank Heimes (fheimes) wrote :

Default Comment by Bridge

Frank Heimes (fheimes) wrote :

Default Comment by Bridge

Frank Heimes (fheimes) on 2021-01-21
Changed in s390-tools (Ubuntu Hirsute):
status: Confirmed → Fix Committed
Mathew Hodson (mhodson) on 2021-01-22
Changed in s390-tools (Ubuntu Focal):
importance: Undecided → Medium
Changed in s390-tools (Ubuntu Groovy):
importance: Undecided → Medium
Changed in s390-tools (Ubuntu Hirsute):
importance: Undecided → Medium
affects: s390-tools → ubuntu-translations
no longer affects: ubuntu-translations
Frank Heimes (fheimes) wrote :
Lukas Märdian (slyon) on 2021-01-25
Changed in s390-tools (Ubuntu Groovy):
status: Triaged → In Progress
Lukas Märdian (slyon) wrote :

I reviewed and uploaded Frank's MP, which contains 3 Focal SRUs, now it needs to be accepted from the queues.
https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug/1902179
https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug/1903984
https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug/1908371

LP: #1903984 still needs to migrate from -proposed in hirsute, before we can mark it "Fix Released", which is currently blocked because of "net-snmp"/"389-ds-base", but we should get this going by applying some hints: https://code.launchpad.net/~slyon/britney/+git/hints-ubuntu/+merge/396858 (https://bugs.launchpad.net/ubuntu/+source/389-ds-base/+bug/1913175)

Changed in s390-tools (Ubuntu Focal):
status: Triaged → In Progress
Frank Heimes (fheimes) on 2021-01-26
Changed in ubuntu-z-systems:
status: Triaged → In Progress

Hello bugproxy, or anyone else affected,

Accepted s390-tools into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-tools/2.14.0-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in s390-tools (Ubuntu Groovy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-groovy
Brian Murray (brian-murray) wrote :

Hello bugproxy, or anyone else affected,

Accepted s390-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-tools/2.12.0-0ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in s390-tools (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Frank Heimes (fheimes) on 2021-01-27
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed

------- Comment From <email address hidden> 2021-01-27 06:45 EDT-------
Feature validated by IBM

Frank Heimes (fheimes) wrote :

Thx for the verification on focal and on groovy. I'm adjusting the tags accordingly ...

tags: added: verification-done verification-done-focal verification-done-groovy
removed: verification-needed verification-needed-focal verification-needed-groovy
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-tools - 2.15.1-0ubuntu6

---------------
s390-tools (2.15.1-0ubuntu6) hirsute; urgency=medium

  * debian/patches/s390-tools-lp1903984-hirsute.patch
    zcryptstats: Fix handling of partial results with many domains
    Thanks to Ingo Franzki (LP: #1903984)

 -- Frank Heimes <email address hidden> Tue, 19 Jan 2021 18:58:26 +0100

Changed in s390-tools (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-tools - 2.14.0-1ubuntu1.1

---------------
s390-tools (2.14.0-1ubuntu1.1) groovy; urgency=medium

  * debian/patches/s390-tools-sru-lp1903984-groovy.patch
    zcryptstats: Fix handling of partial results with many domains
    Thanks to Ingo Franzki (LP: #1903984)
  * debian/patches/s390-tools-sru-lp1898935-groovy.patch:
    chcpumf installed with s390-tools version 2.14.0
    no longer provides version or help information
    backport, actually cherry-pick, from s390-tools v2.15.1 to v2.14
    Thanks to Thomas Richter (LP: #1898935)
  * debian/patches/s390-tools-sru-lp1908371-groovy.patch:
    zipl command isn't working correctly in combination with the -M
    (respectively --mvdump) option.
    cherry-picking 4 commits from s390-tools v2.15.1 to v2.14
    Thanks to Stefan Haberland and Sven Schnelle (LP: #1908371)

 -- Frank Heimes <email address hidden> Tue, 19 Jan 2021 19:23:32 +0100

Changed in s390-tools (Ubuntu Groovy):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for s390-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-tools - 2.12.0-0ubuntu3.2

---------------
s390-tools (2.12.0-0ubuntu3.2) focal; urgency=medium

  * debian/patches/s390-tools-sru-lp1903984-focal.patch
    zcryptstats: Fix handling of partial results with many domains
    Thanks to Ingo Franzki (LP: #1903984)
  * debian/patches/s390-tools-sru-lp1908371-focal.patch:
    zipl command isn't working correctly in combination with the -M
    (respectively --mvdump) option.
    cherry-picking 4 commits from s390-tools v2.15.1 to v2.14
    Thanks to Stefan Haberland and Sven Schnelle (LP: #1908371)
  * debian/patches/s390-tools-sru-lp1902179-focal.patch:
    Enabling IPL (boot) from NVMe devices by adding 2 upstream commits
    from s390-tools v2.13/2.14 as backports to this v2.12 base.
    Thanks to Jason J. Herne (LP: #1902179)

 -- Frank Heimes <email address hidden> Fri, 22 Jan 2021 15:52:19 +0100

Changed in s390-tools (Ubuntu Focal):
status: Fix Committed → Fix Released
Frank Heimes (fheimes) on 2021-01-28
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released

------- Comment From <email address hidden> 2021-02-11 03:35 EDT-------
IBM Bugzilla status->closed, Fix Released with all requested distros

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers