libssl1.1 version 1.1.1-1ubuntu2.1~18.04.2 breaks nginx ssl tests

[Andy Shih]

Many nginx ssl tests pass with libssl1.1 version 1.1.0g-2ubuntu4.3 but fail when libssl1.1 is updated to version 1.1.1-1ubuntu2.1~18.04.2.

Repro steps:
1. Create control Dockerfile:
==========================================
FROM ubuntu:18.04

RUN apt-get update -y && \
  apt-get upgrade -y && \
  apt-get install -y git nginx-core xdg-utils openssl=1.1.0g-2ubuntu4.3 libnet-ssleay-perl=1.84-1build1 libio-socket-ssl-perl=2.056-1 libssl1.1=1.1.0g-2ubuntu4.3

RUN git clone https://github.com/nginx/nginx-tests.git
WORKDIR /nginx-tests

ENV TEST_NGINX_BINARY=/usr/sbin/nginx
ENV TEST_NGINX_MODULES=/usr/lib/nginx/modules

USER www-data

ENTRYPOINT ["prove", "."]
==========================================

2. Run the command in a directory with only the Dockerfile:
docker build -t nginx_image . && docker run --rm -it nginx_image

3. See output:
==========================================
Test Summary Report
-------------------
./grpc_request_buffering.t (Wstat: 512 Tests: 14 Failed: 2)
  Failed tests: 11-12
  Non-zero exit status: 2
./h2_server_tokens.t (Wstat: 1536 Tests: 14 Failed: 6)
  Failed tests: 1-2, 7-8, 10-11
  Non-zero exit status: 6
./upstream_ip_hash_ipv6.t (Wstat: 512 Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
Files=346, Tests=3782, 317 wallclock secs ( 1.87 usr 0.78 sys + 35.84 cusr 16.99 csys = 55.48 CPU)
Result: FAIL
==========================================

4. Create new Dockerfile (only difference is updating libssl1.1):
==========================================
FROM ubuntu:18.04

RUN apt-get update -y && \
  apt-get upgrade -y && \
  apt-get install -y git nginx-core xdg-utils openssl=1.1.0g-2ubuntu4.3 libnet-ssleay-perl=1.84-1build1 libio-socket-ssl-perl=2.056-1 libssl1.1=1.1.1-1ubuntu2.1~18.04.2

RUN git clone https://github.com/nginx/nginx-tests.git
WORKDIR /nginx-tests

ENV TEST_NGINX_BINARY=/usr/sbin/nginx
ENV TEST_NGINX_MODULES=/usr/lib/nginx/modules

USER www-data

ENTRYPOINT ["prove", "."]
==========================================

5. See output
==========================================
Test Summary Report
-------------------
./grpc_request_buffering.t (Wstat: 512 Tests: 14 Failed: 2)
  Failed tests: 11-12
  Non-zero exit status: 2
./h2_server_tokens.t (Wstat: 1536 Tests: 14 Failed: 6)
  Failed tests: 1-2, 7-8, 10-11
  Non-zero exit status: 6
./mail_ssl.t (Wstat: 768 Tests: 22 Failed: 3)
  Failed tests: 3, 5-6
  Non-zero exit status: 3
./proxy_ssl.t (Wstat: 512 Tests: 9 Failed: 2)
  Failed tests: 4-5
  Non-zero exit status: 2
./stream_proxy_ssl.t (Wstat: 512 Tests: 8 Failed: 2)
  Failed tests: 4-5
  Non-zero exit status: 2
./stream_ssl.t (Wstat: 768 Tests: 9 Failed: 3)
  Failed tests: 2, 4-5
  Non-zero exit status: 3
./stream_upstream_zone_ssl.t (Wstat: 768 Tests: 11 Failed: 3)
  Failed tests: 4-5, 9
  Non-zero exit status: 3
./upstream_ip_hash_ipv6.t (Wstat: 512 Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
./upstream_zone_ssl.t (Wstat: 768 Tests: 11 Failed: 3)
  Failed tests: 4-5, 9
  Non-zero exit status: 3
Files=346, Tests=3764, 317 wallclock secs ( 2.00 usr 0.73 sys + 36.49 cusr 16.91 csys = 56.13 CPU)
Result: FAIL
==========================================

New failures: mail_ssl.t, proxy_ssl.t, stream_proxy_ssl.t, stream_ssl.t, stream_upstream_zone_ssl.t, upstream_zone_ssl.t.

[Thomas Ward (teward)]

Default nginx configurations **in Ubuntu** do not enable SSL by default. Have you passed an SSL enabled config to your docker container first?

[Thomas Ward (teward)]

This sounds more like an OpenSSL issue than an NGINX one. Until such investigations are complete to confirm this and rule out NGINX as the problem, I am marking this as 'Incomplete' in NGINX in Ubuntu.

[Andy Shih (aeshih)]

I did not do anything special outside of the steps in my repro to specifically enable SSL in the nginx configs, but should that matter in terms of running these SSL tests? With the Dockerfile in the description as it is, I can already see in the test output that SSL tests are running and succeeding with libssl1.1 on version 1.1.0g-2ubuntu4.3 and failing on version 1.1.1-1ubuntu2.1~18.04.2.

Here is one example (mail_ssl.t):

Dockerfile with libssl1.1=1.1.0g-2ubuntu4.3:
www-data@8c157d70f8a2:/nginx-tests$ prove mail_ssl.t
mail_ssl.t..ok
All tests successful.
Files=1, Tests=22, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.15 cusr 0.27 csys = 0.44 CPU)
Result: PASS

Dockerfile with libssl1.1=1.1.1-1ubuntu2.1~18.04.2:
www-data@2a68517d6b29:/nginx-tests$ prove mail_ssl.t
mail_ssl.t .. 140319190012352:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/var/www/.rnd
140531605090752:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/var/www/.rnd
mail_ssl.t .. 1/22
# Failed test 'builtin session reused'
# at mail_ssl.t line 187.
# got: '0'
# expected: '1'

# Failed test 'builtin size session reused'
# at mail_ssl.t line 199.
# got: '0'
# expected: '1'

# Failed test 'shared session reused'
# at mail_ssl.t line 205.
# got: '0'
# expected: '1'
# Looks like you failed 3 tests of 22.
mail_ssl.t .. Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/22 subtests

Test Summary Report
------------------- _ssl.t (Wstat: 768 Tests: 22 Failed: 3)
  Failed tests: 3, 5-6
  Non-zero exit status: 3
Files=1, Tests=22, 1 wallclock secs ( 0.02 usr 0.01 sys + 0.16 cusr 0.25 csys = 0.44 CPU)
Result: FAIL

[Andy Shih (aeshih)]

Another thing to mention is that the new failures are all due to the same error as in the previous comment. It seems like the failing tests all expect the session to be reused, and fails.

[Fahad (fahad-alsaidi)]

After upgrade to OpenSSL 1.1.1 I get this err_ssl_version_interference error in chrome.
running nginx -V I got this:

nginx -V
nginx version: nginx/1.14.0 (Ubuntu)
built with OpenSSL 1.1.0g 2 Nov 2017 (running with OpenSSL 1.1.1 11 Sep 2018)
TLS SNI support enabled

so I don't know where the problem from. please see
https://trac.nginx.org/nginx/ticket/1654

[Sebastien Bacher (seb128)]

Could you try if the followup SRU made a difference there?
https://bugs.launchpad.net/ubuntu/+source/openssl/1.1.1-1ubuntu2.1~18.04.3

[Andy Shih (aeshih)]

Looks like the update to 1.1.1-1ubuntu2.1~18.04.4 resolved the issue! The listed failing tests above are now passing.

I don't see a way I can close out an issue.