wget crashes with openssl-ibmca

Bug #1543455 reported by Dimitri John Ledkov
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
bugproxy
openssl-ibmca (Ubuntu)
Fix Released
High
Dimitri John Ledkov
Xenial
Fix Released
High
Dimitri John Ledkov
Zesty
Won't Fix
High
Unassigned
Artful
Fix Released
High
Dimitri John Ledkov

Bug Description

[Impact]

 * wget fails to download anything over https, when libssl is configured with openssl-ibmca engine on s390x.

[Test Case]

 * configure and enable openssl-ibmca
 * wget https://launchpad.net

[Regression Potential]

 * the engine code path, used to refuse initializing multiple times. However, that action is out of engines control, instead initialization should becomes a no-op upon multiple calls. Upstream patch, well tested in subsequent releases.

[Other Info]

 * Original bug report.

With openssl-ibmca enabled, wget crashes downloading things from launchpad librarian.

actual bug in openssl-ibmca package.

Frank Heimes (fheimes)
Changed in wget (Ubuntu):
status: New → Confirmed
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
assignee: nobody → bugproxy (bugproxy)
bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-156507 severity-high targetmilestone-inin1704
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-07-10 08:21 EDT-------
As of now I am not aware of any issue with ibmca/libica.
Can you please provide some more info ?
The libica version, the openssl-ibmca version and maybe a callstack of the failure would be nice. If possible, it would help to have some verbose output from wget (maybe the cipher used), or is it crashing at the very beginning ? Then a strace would help.
Thanks
H. Freudenberger

Revision history for this message
Frank Heimes (fheimes) wrote :
Download full text (27.3 KiB)

Sorry, this LP entry was so far just a stub.

But here are the details:

$ openssl engine
(dynamic) Dynamic engine loading support
(ibmca) Ibmca hardware engine support
$ openssl engine -c
(dynamic) Dynamic engine loading support
(ibmca) Ibmca hardware engine support
 [RSA, DSA, DH, RAND, DES-ECB, DES-CBC, DES-OFB, DES-CFB, DES-EDE3, DES-EDE3-CBC, DES-EDE3-OFB, DES-EDE3-CFB, AES-128-ECB, AES-192-ECB, AES-256-ECB, AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-128-OFB, AES-192-OFB, AES-256-OFB, AES-128-CFB, AES-192-CFB, AES-256-CFB, SHA1, SHA256, SHA512]

$ apt-cache policy openssl-ibmca
openssl-ibmca:
  Installed: 1.3.0-0ubuntu4
  Candidate: 1.3.0-0ubuntu4
  Version table:
 *** 1.3.0-0ubuntu4 500
        500 http://us.ports.ubuntu.com/ubuntu-ports zesty/universe s390x Packages
        100 /var/lib/dpkg/status

$ apt-cache policy libica-utils
libica-utils:
  Installed: 3.0.2-1
  Candidate: 3.0.2-1
  Version table:
 *** 3.0.2-1 500
        500 http://us.ports.ubuntu.com/ubuntu-ports zesty/universe s390x Packages
        100 /var/lib/dpkg/status

$ apt-cache policy libica3
libica3:
  Installed: 3.0.2-1
  Candidate: 3.0.2-1
  Version table:
 *** 3.0.2-1 500
        500 http://us.ports.ubuntu.com/ubuntu-ports zesty/universe s390x Packages
        100 /var/lib/dpkg/status

$ apt-cache policy wget
wget:
  Installed: 1.18-2ubuntu1
  Candidate: 1.18-2ubuntu1
  Version table:
 *** 1.18-2ubuntu1 500
        500 http://us.ports.ubuntu.com/ubuntu-ports zesty/main s390x Packages
        100 /var/lib/dpkg/status

$ wget http://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb
--2017-07-10 10:52:42-- http://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb
Resolving squid.internal (squid.internal)... 91.189.89.216
Connecting to squid.internal (squid.internal)|91.189.89.216|:3128... connected.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: https://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb [following]
--2017-07-10 10:52:43-- https://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb
Segmentation fault (core dumped)
$ mv /var/crash/_usr_bin_wget.1000.crash /var/crash/_usr_bin_wget.1000.crash_1st
$ wget https://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb
--2017-07-10 10:53:04-- https://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb
Segmentation fault (core dumped)
$ mv /var/crash/_usr_bin_wget.1000.crash /var/crash/_usr_bin_wget.1000.crash_2nd
$ wget --verbose https://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/12768113/+files/perftest_3.4+0.6.gc3435c2-1ubuntu1_s390x.deb
--2017-07-10 10:53:19-- https://bugs.launchpad.net/ubuntu/+source/perftest/3.4+0.6.gc3435c2-1ubuntu1/+build/...

Revision history for this message
Frank Heimes (fheimes) wrote :
Revision history for this message
Frank Heimes (fheimes) wrote :
Revision history for this message
Frank Heimes (fheimes) wrote :
Revision history for this message
Frank Heimes (fheimes) wrote :
Revision history for this message
Frank Heimes (fheimes) wrote :
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-07-19 10:10 EDT-------
I've been able to recreate this.
investigating ...

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → Confirmed
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-07-20 05:36 EDT-------
Catched the reason:
All the internal function pointers in libibmca are NULL. So the sha1 call jumps over zero ;-). The reason why all the function pointers are NULL is simple because the ibmca engine init function ibmca_init() is called 3 (!!!) times.
The start of this function is:

static int ibmca_init(ENGINE * e)
{
static int init = 0;

if (ibmca_dso != NULL && init >= 2) {
IBMCAerr(IBMCA_F_IBMCA_INIT, IBMCA_R_ALREADY_LOADED);
goto err;
}

so init values >= 2 will cause the code to jump to the err label where all internal function pointers are set to NULL. Tracing shows this very clear. The first init works fine, the second similar but the third init now decides to clear the function pointers and that's it.

Solution: Calling init more than once is certainly a feature :-). However, a rework of the code like this:

static int ibmca_init(ENGINE * e)
{
static int init = 0;

if (init > 0)
return 1;

and everything is fine. So openssl call the engine's init function as often as wanted it does no harm.

I'll attach a patch. Paulo please review the patch, maybe comment and rework and integrate this or another solution into the ibmca engine.
regards
H.Freudenberger

Revision history for this message
bugproxy (bugproxy) wrote : openssl-ibmca engine init cout patch

------- Comment (attachment only) From <email address hidden> 2017-07-20 05:49 EDT-------

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Confirmed → In Progress
Changed in wget (Ubuntu):
status: Confirmed → In Progress
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-07-25 10:17 EDT-------
Upstream commit 6bae168 fixes this issue.

https://github.com/opencryptoki/openssl-ibmca/commit/6bae168d2f4f678d7c6d0c201a4230e14d2f912e

Revision history for this message
bugproxy (bugproxy) wrote : _usr_bin_wget.1000.crash_1st

Default Comment by Bridge

Revision history for this message
bugproxy (bugproxy) wrote : _usr_bin_wget.1000.crash_2nd

Default Comment by Bridge

Revision history for this message
bugproxy (bugproxy) wrote : _usr_bin_wget.1000.crash_3rd

Default Comment by Bridge

Revision history for this message
bugproxy (bugproxy) wrote : _usr_bin_wget.1000.crash_4th

Default Comment by Bridge

Revision history for this message
bugproxy (bugproxy) wrote : openssl.cnf

Default Comment by Bridge

Revision history for this message
bugproxy (bugproxy) wrote : openssl-ibmca engine init cout patch

------- Comment (attachment only) From <email address hidden> 2017-07-20 05:49 EDT-------

affects: wget (Ubuntu) → openssl-ibmca (Ubuntu)
Changed in openssl-ibmca (Ubuntu):
status: In Progress → Fix Committed
milestone: none → ubuntu-17.08
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openssl-ibmca - 1.3.0-0ubuntu5

---------------
openssl-ibmca (1.3.0-0ubuntu5) artful; urgency=medium

  * Apply upstream patch to resolve crashes when libssl attempts to
    initialise engine a few times too many. LP: #1543455

 -- Dimitri John Ledkov <email address hidden> Wed, 26 Jul 2017 08:48:51 +0100

Changed in openssl-ibmca (Ubuntu):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Released → In Progress
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-08-02 09:31 EDT-------
IBM Bugzilla status -> closed. fixed with artful. to be SRUed on xenial,zesty

Mathew Hodson (mhodson)
Changed in openssl-ibmca (Ubuntu Xenial):
importance: Undecided → High
Changed in openssl-ibmca (Ubuntu Zesty):
importance: Undecided → High
Changed in openssl-ibmca (Ubuntu Artful):
importance: Undecided → High
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
importance: Undecided → High
Changed in openssl-ibmca (Ubuntu Zesty):
status: New → Won't Fix
Changed in openssl-ibmca (Ubuntu Xenial):
status: New → Triaged
assignee: nobody → Dimitri John Ledkov (xnox)
tags: added: id-5a7c3ab43ac40a5aabeb697d
tags: added: id-5a969d9f612b080a49b18982
description: updated
Changed in openssl-ibmca (Ubuntu Xenial):
status: Triaged → In Progress
Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted openssl-ibmca into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl-ibmca/1.3.0-0ubuntu2.16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in openssl-ibmca (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
Revision history for this message
Frank Heimes (fheimes) wrote :

The above package could be successfully tested on latest xenial version - see attachment.

Tags are adjusted accordingly.

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Frank Heimes (fheimes)
tags: added: universe
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openssl-ibmca - 1.3.0-0ubuntu2.16.04.2

---------------
openssl-ibmca (1.3.0-0ubuntu2.16.04.2) xenial; urgency=medium

  * Apply upstream patch to resolve crashes when libssl attempts to
    initialise engine a few times too many. LP: #1543455

 -- Dimitri John Ledkov <email address hidden> Tue, 24 Jul 2018 16:21:15 +0100

Changed in openssl-ibmca (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for openssl-ibmca has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.