Do not support OpenSSL 3

Bug #1964025 reported by Lucas Kanashiro
96
This bug affects 15 people
Affects Status Importance Assigned to Milestone
ruby-bcrypt-pbkdf (Ubuntu)
Status tracked in Kinetic
Jammy
Confirmed
Undecided
Unassigned
Kinetic
Invalid
Undecided
Unassigned
ruby-net-ssh (Ubuntu)
Status tracked in Kinetic
Jammy
Fix Released
Critical
Lucas Kanashiro
Kinetic
Fix Released
Critical
Lucas Kanashiro
vagrant (Ubuntu)
Status tracked in Kinetic
Jammy
Confirmed
Undecided
Unassigned
Kinetic
Triaged
Undecided
Unassigned

Bug Description

[Impact]

The ruby-net-ssh package in jammy fails most operations due to its incompatibility with OpenSSL 3.0, thus breaking most of its reverse-dependencies.

[Test case]

Amend the following one-liner for a host to which you have SSH access via pubkey:

ruby -e "require 'net/ssh'; Net::SSH.start('$REMOTE_HOST', '$REMOTE_USER') do |ssh| puts ssh.exec!('hostname') end"

It currently fails with the following error:
/usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/buffer.rb:316:in `set_key': rsa#set_key= is incompatible with OpenSSL 3.0 (OpenSSL::PKey::PKeyError

[Where problems could occur]

Even though the package is currently unusable, the fix could introduce problems via some subtle missuses of the new OpenSSL APIs.

[Original report]
Upstream still does not support OpenSSL 3, and due to that a bunch of tests are failing. Fedora has been seeing the same problem and they filed a bug upstream:

https://github.com/net-ssh/net-ssh/issues/843

Related branches

Changed in ruby-net-ssh (Ubuntu):
importance: Undecided → High
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

This issue is currently blocking ruby2.7 removal from Jammy. The removal requires the migration of ruby-bcrypt-pbkdf in jammy-proposed:

$ reverse-depends src:ruby2.7
Reverse-Depends
* ruby-bcrypt-pbkdf (for libruby2.7)

Packages without architectures listed are reverse-dependencies in: amd64, arm64, armhf, ppc64el, s390x

And ruby-net-ssh is the only regression blocking the ruby-bcrypt-pbkdf migration.

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :
Download full text (6.1 KiB)

It is also impacting vagrant already in Jammy (the release pocket). The following does not work because of this issue:

$ vagrant init debian/buster64
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
$ vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Box 'debian/buster64' could not be found. Attempting to find and install...
    default: Box Provider: libvirt
    default: Box Version: >= 0
==> default: Loading metadata for box 'debian/buster64'
    default: URL: https://vagrantcloud.com/debian/buster64
==> default: Adding box 'debian/buster64' (v10.20211230.1) for provider: libvirt
    default: Downloading: https://vagrantcloud.com/debian/boxes/buster64/versions/10.20211230.1/providers/libvirt.box
==> default: Successfully added box 'debian/buster64' (v10.20211230.1) for 'libvirt'!
==> default: Uploading base box image as volume into Libvirt storage...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default: -- Name: test_default
==> default: -- Description: Source: /tmp/test/Vagrantfile
==> default: -- Domain type: kvm
==> default: -- Cpus: 1
==> default: -- Feature: acpi
==> default: -- Feature: apic
==> default: -- Feature: pae
==> default: -- Clock offset: utc
==> default: -- Memory: 512M
==> default: -- Management MAC:
==> default: -- Loader:
==> default: -- Nvram:
==> default: -- Base box: debian/buster64
==> default: -- Storage pool: default
==> default: -- Image(): /var/lib/libvirt/images/test_default.img, 20G
==> default: -- Disk driver opts: cache='default'
==> default: -- Kernel:
==> default: -- Initrd:
==> default: -- Graphics Type: vnc
==> default: -- Graphics Port: -1
==> default: -- Graphics IP: 127.0.0.1
==> default: -- Graphics Password: Not defined
==> default: -- Video Type: cirrus
==> default: -- Video VRAM: 9216
==> default: -- Video 3D accel: false
==> default: -- Sound Type:
==> default: -- Keymap: en-us
==> default: -- TPM Backend: passthrough
==> default: -- TPM Path:
==> default: -- INPUT: type=mouse, bus=ps2
==> default: Creating shared folders metadata...
==> default: Starting domain.
==> default: Waiting for domain to get an IP address...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 192.168.121.18:22
    default: SSH username: vagrant
    default: SSH auth method: private key
==> default: Removing domain...
==> default: Deleting the machine folder
/usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/kex/ecdh_sha2_nistp256.rb:21:in `generate_key!': pkeys are immutable on OpenSSL 3.0 (OpenSSL::PKey::PKeyError)
 from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport...

Read more...

Changed in ruby-net-ssh (Ubuntu):
status: New → In Progress
importance: High → Critical
Simon Chopin (schopin)
tags: added: transition-openssl3-jj
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

The solution for this issue reported in ruby/openssl will fix part of the net-ssh issue:

https://github.com/ruby/openssl/issues/498

Revision history for this message
Steve Langasek (vorlon) wrote :

ruby-defaults 3.0 migrating to release means that ruby-net-ssh is now broken in the release pocket, so a baseline retest has let ruby-bcrypt-pbkdf so closing that task.

Changed in ruby-bcrypt-pbkdf (Ubuntu):
status: New → Invalid
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

I also filed this issue which involves some ruby-net-ssh test failures:

https://github.com/ruby/openssl/issues/500

Changed in vagrant (Ubuntu):
status: New → Triaged
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

With the version in archive this is the summary of the execution of the test suite during the build:

1494 runs, 3679 assertions, 28 failures, 395 errors, 0 skips

Applying the WIP patch (attached) the summary is:

1494 runs, 3964 assertions, 16 failures, 332 errors, 0 skips

tags: added: patch
Revision history for this message
Luis Alberto Pabón (copong) wrote :

This currently breaks Vagrant.

Simon Chopin (schopin)
tags: added: fr-2166
Revision history for this message
Simon Chopin (schopin) wrote (last edit ):

Here's my WIP patch, building upon Lucas' patch. If I run the tests with OPENSSL_CONF pointing to the ssl.conf file in this patch (which has the legacy provider enabled) I get

1504 runs, 4940 assertions, 4 failures, 23 errors, 0 skips

Deduplicating the errors, there are 4 types of failures that I can see:

EVP_PKEY_derive_set_peer failures, from the test_diffie_hellman_group14_sha1.rb (7 or 8 failures) -> This has me completely puzzled, as the tests work fine for the group1 data.

OpenSSL::PKey::PKeyError: EVP_PKEY_keygen: bad ffc parameters (most errors)
->I haven't looked into this one just yet

Authentication::TestKeyManager#test_identities_with_ecdsa_should_load_from_agent:
OpenSSL::PKey::PKeyError: pkeys are immutable on OpenSSL 3.0
-> This one is similar to many other already fixed

OpenSSL::PKey::DSAError: incorrect pkey type: dhpublicnumber
-> I looked quickly into it. I suspect it's a bug in the OpenSSL Ruby bindings?

I'll keep at it tomorrow.

NB: I'm running the test suite manually from upstream git with the sole Debian patch applied, which explains why the total number of tests differs.

Revision history for this message
Simon Chopin (schopin) wrote :

Attached is the newest version of the patch, which solves all failures mentioned above except for the EVP_PKEY_derive_set_peer diffie-hellman group 14 one:

1504 runs, 5067 assertions, 4 failures, 3 errors, 0 skips

Next steps are

* split the patch into smaller patches to ease upstream inclusion
* automatically load the custom ssl config with legacy providers if OpenSSL 3.0 is detected
* try to mock up the group 14 issue in a C PoC to get some OpenSSL upstream eyeballs on the problem

Revision history for this message
Melvin Loos (melvin-7) wrote :

Any update on this?

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ruby-bcrypt-pbkdf (Ubuntu Jammy):
status: New → Confirmed
Changed in ruby-net-ssh (Ubuntu Jammy):
status: New → Confirmed
Changed in vagrant (Ubuntu Jammy):
status: New → Confirmed
Simon Chopin (schopin)
description: updated
Changed in ruby-net-ssh (Ubuntu Jammy):
importance: Undecided → Critical
Changed in ruby-net-ssh (Ubuntu Kinetic):
assignee: nobody → Lucas Kanashiro (lucaskanashiro)
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

ruby-net-ssh/1:7.0.0~beta1-2 has the fix and it is in kinetic-proposed right now, waiting for the reverse dependencies tests to be executed to migrate to the release pocket. In the meantime, I am going to backport the needed changes to Jammy.

Changed in ruby-net-ssh (Ubuntu Kinetic):
status: In Progress → Fix Committed
Changed in ruby-net-ssh (Ubuntu Jammy):
assignee: nobody → Lucas Kanashiro (lucaskanashiro)
status: Confirmed → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Lucas, or anyone else affected,

Accepted ruby-net-ssh into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ruby-net-ssh/1:6.1.0-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ruby-net-ssh (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

# Test Plan

$ lxc launch ubuntu-daily:jammy ruby-net-ssh-test
$ lxc shell ruby-net-ssh-test
# apt update && apt upgrade -y
<enable jammy-proposed>
# apt install -y ruby-net-ssh
# dpkg -l | grep ruby-net-ssh
ii ruby-net-ssh 1:6.1.0-2ubuntu0.1 all Ruby implementation of the SSH protocol
<try to access the host machine>
# ruby -e "require 'net/ssh'; Net::SSH.start('10.191.226.1', 'lucas') do |ssh| puts ssh.exec!('hostname') end"
lucas@10.191.226.1's password:
poseidon

No error as expected. I'll let other affected people test the package in jammy-proposed before flipping the tags to done.

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :
Download full text (4.1 KiB)

Since no one replied yet, I tried to setup a VM using vagrant as I described in comment #2, and it worked as expected:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
$ dpkg -l | grep ruby-net-ssh
ii ruby-net-ssh 1:6.1.0-2ubuntu0.1 all Ruby implementation of the SSH protocol
$ vagrant init debian/buster64
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
$ vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'debian/buster64' version '10.20211230.1' is up to date...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default: -- Name: test_default
==> default: -- Description: Source: /tmp/test/Vagrantfile
==> default: -- Domain type: kvm
==> default: -- Cpus: 1
==> default: -- Feature: acpi
==> default: -- Feature: apic
==> default: -- Feature: pae
==> default: -- Clock offset: utc
==> default: -- Memory: 512M
==> default: -- Management MAC:
==> default: -- Loader:
==> default: -- Nvram:
==> default: -- Base box: debian/buster64
==> default: -- Storage pool: default
==> default: -- Image(): /var/lib/libvirt/images/test_default.img, 20G
==> default: -- Disk driver opts: cache='default'
==> default: -- Kernel:
==> default: -- Initrd:
==> default: -- Graphics Type: vnc
==> default: -- Graphics Port: -1
==> default: -- Graphics IP: 127.0.0.1
==> default: -- Graphics Password: Not defined
==> default: -- Video Type: cirrus
==> default: -- Video VRAM: 9216
==> default: -- Video 3D accel: false
==> default: -- Sound Type:
==> default: -- Keymap: en-us
==> default: -- TPM Backend: passthrough
==> default: -- TPM Path:
==> default: -- INPUT: type=mouse, bus=ps2
==> default: Creating shared folders metadata...
==> default: Starting domain.
==> default: Waiting for domain to get an IP address...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 192.168.121.3:22
    default: SSH username: vagrant
    default: SSH auth method: private key
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present......

Read more...

tags: added: verification-done verification-done-jammy
removed: verification-needed verification-needed-jammy
Revision history for this message
Marco Roeland (marcoroeland) wrote :

I installed ruby-net-ssh version 1:6.1.0-2ubuntu0.1 from jammy-proposed as suggested above.
With this package installed I can again use vagrant to install a "generic/centos7" vagrant box, using the default libvirt provider. Thanks!

tags: verification-done verification-done-jammy

Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for ruby-net-ssh has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ruby-net-ssh - 1:6.1.0-2ubuntu0.1

---------------
ruby-net-ssh (1:6.1.0-2ubuntu0.1) jammy; urgency=medium

  * d/p/openssl-3/*.patch: backport upstream patches to support OpenSSL 3
    (LP: #1964025).
  * d/ruby-tests.rake: use custom OpenSSL config file if using OpenSSL 3.

 -- Lucas Kanashiro <email address hidden> Mon, 09 May 2022 18:44:20 -0300

Changed in ruby-net-ssh (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Alexander Adam (7ql6) wrote :

Should these updates appear automatically at one point or do I have to activate some repos or so?

I'm still having the same issues like others as well:

https://stackoverflow.com/questions/71987581/openssl-3-0-error-when-booting-vagrantbox

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

No, you do not need to enable any repository. ruby-net-ssh/1:6.1.0-2ubuntu0.1 landed in jammy-updates which should be enabled by default on your system.

The link to the stackoverflow page you sent is old, and at that time it was broken. This fix was even acknowledged on an upstream issue: https://github.com/hashicorp/vagrant/issues/12751#issuecomment-1132239703

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

This fix is also released in kinetic, ruby-net-ssh/1:7.0.0~beta1-2 contains the fix.

Changed in ruby-net-ssh (Ubuntu Kinetic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.