Segfault on AArch64 caused by OpenSSL affecting numerous packages

[Nazar Mokrynskyi]

OpenSSL causes crashes when reaching to some URLs on AArch64 platform, affecting Ubuntu, but not Fedora for instance.

Initially reported in https://mediasoup.discourse.group/t/mediasoup-worker-default-make-failed/3647/12, more details and reproductions in https://github.com/mesonbuild/meson/issues/9690

Affects curl, wget, python and probably everything else.

[Alex Murray (alexmurray)]

FWIW I can't reproduce this on a RPi 4 running the aarch64/arm64 Ubuntu 20.04 LTS image:

ubuntu@rpi4:~$ wget https://wrapdb.mesonbuild.com/v2/libuv_1.42.0-1/get_patch
--2021-12-07 05:50:01-- https://wrapdb.mesonbuild.com/v2/libuv_1.42.0-1/get_patch
Resolving wrapdb.mesonbuild.com (wrapdb.mesonbuild.com)... 138.201.247.118
Connecting to wrapdb.mesonbuild.com (wrapdb.mesonbuild.com)|138.201.247.118|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/mesonbuild/wrapdb/releases/download/libuv_1.42.0-1/libuv_1.42.0-1_patch.zip [following]
--2021-12-07 05:50:03-- https://github.com/mesonbuild/wrapdb/releases/download/libuv_1.42.0-1/libuv_1.42.0-1_patch.zip
Resolving github.com (github.com)... 13.236.229.21
Connecting to github.com (github.com)|13.236.229.21|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/236250352/46c49bec-514b-4411-afe8-46ac8cb2e82f?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211207%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211207T054758Z&X-Amz-Expires=300&X-Amz-Signature=504c83b4d0c3567dc2f509362714a5b5709951655612c5665ca7d3e1f09050c5&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=236250352&response-content-disposition=attachment%3B%20filename%3Dlibuv_1.42.0-1_patch.zip&response-content-type=application%2Foctet-stream [following]
--2021-12-07 05:50:03-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/236250352/46c49bec-514b-4411-afe8-46ac8cb2e82f?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211207%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211207T054758Z&X-Amz-Expires=300&X-Amz-Signature=504c83b4d0c3567dc2f509362714a5b5709951655612c5665ca7d3e1f09050c5&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=236250352&response-content-disposition=attachment%3B%20filename%3Dlibuv_1.42.0-1_patch.zip&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5146 (5.0K) [application/octet-stream]
Saving to: ‘get_patch’

get_patch 100%[=========================================================================================================================================>] 5.03K --.-KB/s in 0.009s

2021-12-07 05:50:04 (590 KB/s) - ‘get_patch’ saved [5146/5146]

ubuntu@rpi4:~$ dpkg -l openssl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-=================-============-====================================================
ii openssl 1.1.1f-1ubuntu2.9 arm64 Secure Sockets Layer toolkit - cryptographic utility
ubun...

[Nazar Mokrynskyi (nazar-pc)]

Not sure why it doesn't reproduce on RPi 4, but I guess as described on GitHub, might be because of assembly instructions that work on RPi 4 and not in some virtualized environments (both binfmt and Docker under macOS M1 are virtualization technologies).

This is on my x86-64 Linux system:
nazar-pc@nazar-pc:~> docker run --rm -it -u root --platform linux/arm64 ubuntu:20.04
root@6fbdb0317cd7:/# apt-get update
Get:1 http://ports.ubuntu.com/ubuntu-ports focal InRelease [265 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease [114 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease [108 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease [114 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports focal/universe arm64 Packages [11.1 MB]
Get:6 http://ports.ubuntu.com/ubuntu-ports focal/restricted arm64 Packages [1317 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports focal/multiverse arm64 Packages [139 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports focal/main arm64 Packages [1234 kB]
Get:9 http://ports.ubuntu.com/ubuntu-ports focal-updates/universe arm64 Packages [1036 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports focal-updates/multiverse arm64 Packages [9055 B]
Get:11 http://ports.ubuntu.com/ubuntu-ports focal-updates/restricted arm64 Packages [3529 B]
Get:12 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 Packages [1244 kB]
Get:13 http://ports.ubuntu.com/ubuntu-ports focal-backports/universe arm64 Packages [21.6 kB]
Get:14 http://ports.ubuntu.com/ubuntu-ports focal-backports/main arm64 Packages [50.0 kB]
Get:15 http://ports.ubuntu.com/ubuntu-ports focal-security/restricted arm64 Packages [3291 B]
Get:16 http://ports.ubuntu.com/ubuntu-ports focal-security/main arm64 Packages [871 kB]
Get:17 http://ports.ubuntu.com/ubuntu-ports focal-security/multiverse arm64 Packages [3242 B]
Get:18 http://ports.ubuntu.com/ubuntu-ports focal-security/universe arm64 Packages [759 kB]
Fetched 17.1 MB in 6s (3070 kB/s)
Reading package lists... Done
root@6fbdb0317cd7:/# apt-get install -y --no-install-recommends ca-certificates curl
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libasn1-8-heimdal libbrotli1 libcurl4 libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common libnghttp2-14 libpsl5 libroken18-heimdal librtmp1
  libsasl2-2 libsasl2-modules-db libsqlite3-0 libssh-4 libssl1.1 libwind0-heimdal openssl
Suggested packages:
  krb5-doc krb5-user
Recommended packages:
  krb5-locales publicsuffix libsasl2-modules
The following NEW packages will be installed:
  ca-certificates curl libasn1-8-heimdal libbrotli1 libcurl4 libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common libnghttp2-14 libpsl5
  libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules-db libsql...

[Seth Arnold (seth-arnold)]

This comment looks promising https://github.com/mesonbuild/meson/issues/9690#issuecomment-986872688

It identifies https://github.com/openssl/openssl/pull/13256 and https://github.com/openssl/openssl/pull/13218 as candidate fixes.

[Simon Chopin (schopin)]

Sounds suspiciously similar to https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1951279 but I'd like confirmation that the situation described in the meson issue comment (TLS 1.2 with ECDHE-RSA-CHACHA20-POLY1305) also applies there before marking as duplicate.

[David Hess (dhess-8)]

I confirmed https://github.com/openssl/openssl/pull/13256 does indeed appear to be the source of the problem for me. I added all of the details to 1951279 here:

https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1951279/comments/15

and

https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1951279/comments/16

Regarding reproducibility, the bug both trashes the stack (which makes it difficult to narrow down) and only happens on v8.3 64 bit ARM processors (which is why some Arm 64 bit folks get false negatives).