openssl 1.0.1f 'ssl handshake failure' connection failure

[Jared Kipe]

When attempting 'curl' or 'openssl s_client' I have been getting "error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure"

openssl version
OpenSSL 1.0.1f 6 Jan 2014

examples:

```Normal connect
# openssl s_client -connect centinel1000.cardinalcommerce.com:443 -showcerts
CONNECTED(00000003)
140148901013152:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 317 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

```Explicit SSLv3
# openssl s_client -connect centinel1000.cardinalcommerce.com:443 -showcerts -ssl3
CONNECTED(00000003)
depth=2 C = US, O = "thawte, Inc.", OU = Certification Services Division, OU = "(c) 2006 thawte, Inc. - For authorized use only", CN = thawte Primary Root CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=Ohio/L=Mentor/O=CardinalCommerce Corporation/CN=*.cardinalcommerce.com
   i:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
-----BEGIN CERTIFICATE-----
MIIEqjCCA5KgAwIBAgIQfI2U+db8heb8kd3m/BmmITANBgkqhkiG9w0BAQUFADA8
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMuMRYwFAYDVQQDEw1U
aGF3dGUgU1NMIENBMB4XDTEzMDMxOTAwMDAwMFoXDTE1MDQxODIzNTk1OVowdTEL
MAkGA1UEBhMCVVMxDTALBgNVBAgTBE9oaW8xDzANBgNVBAcUBk1lbnRvcjElMCMG
A1UEChQcQ2FyZGluYWxDb21tZXJjZSBDb3Jwb3JhdGlvbjEfMB0GA1UEAxQWKi5j
YXJkaW5hbGNvbW1lcmNlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAMUnIwZ0yEJa80hN4sta/wbr04ogq9XwlY7V7iWiLlfoP/wfpccPt/282+AN
oySwuxMWE5EPHC54WXjCowoj3Kdq7fuH11R6DBoXGfuhIJ9l9L187hEPPk6bLq3H
F1diHFxGYT0zCNshm7w7Qe/LmQ8N0WSUs21KuB/WZxEts7sIYi4xY/Ig1Mbt6dVV
z3w3mfSqpXmdZa5ht7/QUEy3/04uGlSXAN01BVmxHbZeM5epocUCt/TwhtUzRb+N
9S4VEe3kzP8Oz8Wphg1CueP5yH9nRQTzLct5wCBC5+N+RjdadhuRm4FPgbsO+sX4
LHQ1jgE6CTqYquyYAeXuvdOqz6kCAwEAAaOCAW0wggFpMCEGA1UdEQQaMBiCFiou
Y2FyZGluYWxjb21tZXJjZS5jb20wCQYDVR0TBAIwADBCBgNVHSAEOzA5MDcGCmCG
SAGG+EUBBzYwKTAnBggrBgEFBQcCARYbaHR0cHM6Ly93d3cudGhhd3RlLmNvbS9j
cHMvMA4GA1UdDwEB/wQEAwIFoDAfBgNVHSMEGDAWgBSnooO7NEVAPfzVME8SuT6h
AZ/22zA6BgNVHR8EMzAxMC+gLaArhilodHRwOi8vc3ZyLW92LWNybC50aGF3dGUu
Y29tL1RoYXd0ZU9WLmNybDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
aQYIKwYBBQUHAQEEXTBbMCIGCCsGAQUFBzABhhZodHRwOi8vb2NzcC50aGF3dGUu
Y29tMDUGCCsGAQUFBzAChilodHRwOi8vc3ZyLW92LWFpYS50aGF3dGUuY29tL1Ro
YXd0ZU9WLmNlcjANBgkqhkiG9w0BAQUFAAOCAQEAQKaqABf0+hz+MkHwn6HhnZ6T
3D7u3a3ePrQQgtZWFo+7A5s0C+UA6SBRcvEZDRP7TMZaU+Ft+stglyby33b3koTQ
2X1F484ncBJGyiOBk0M/KQHIsQGUmeXKNLfZlqXhicbT2nq7SktybPR0rsPJoiqN
gR8pNlHseb1aHM79NcV9IbpW8B71fEMFQRd7sUvmxGizqOneG4nGXCk04CRRy5H3
raU6Xb2CRi5UdjsJPWjLjLDQZBF5aA0IgOZDi7BghU9cy+P4t2PdBBvPP0ctWI3O
LYMF6figGyaw3kCLi4epJ0ZA4ayg8R7KrDNGA7oWI2roknlJd0YEDE3z0Fg2JA==
-----END CERTIFICATE-----
 1 s:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
   i:/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2006 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA
-----BEGIN CERTIFICATE-----
MIIEbDCCA1SgAwIBAgIQTV8sNAiyTCDNbVB+JE3J7DANBgkqhkiG9w0BAQUFADCB
qTELMAkGA1UEBhMCVVMxFTATBgNVBAoTDHRoYXd0ZSwgSW5jLjEoMCYGA1UECxMf
Q2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjE4MDYGA1UECxMvKGMpIDIw
MDYgdGhhd3RlLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNV
BAMTFnRoYXd0ZSBQcmltYXJ5IFJvb3QgQ0EwHhcNMTAwMjA4MDAwMDAwWhcNMjAw
MjA3MjM1OTU5WjA8MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMu
MRYwFAYDVQQDEw1UaGF3dGUgU1NMIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEAmeSFW3ZJfS8F2MWsyMip09yY5tc0pi8M8iIm2KPJFEyPBaRF6BQM
WJAFGrfFwQalgK+7HUlrUjSIw1nn72vEJ0GMK2Yd0OCjl5gZNEtB1ZjVxwWtouTX
7QytT8G1sCH9PlBTssSQ0NQwZ2ya8Q50xMLciuiX/8mSrgGKVgqYMrAAI+yQGmDD
7bs6yw9jnw1EyVLhJZa/7VCViX9WFLG3YR0cB4w6LPf/gN45RdWvGtF42MdxaqMZ
pzJQIenyDqHGEwNESNFmqFJX1xG0k4vlmZ9d53hR5U32t1m0drUJN00GOBN6HAiY
XMRISstSoKn4sZ2Oe3mwIC88lqgRYke7EQIDAQABo4H7MIH4MDIGCCsGAQUFBwEB
BCYwJDAiBggrBgEFBQcwAYYWaHR0cDovL29jc3AudGhhd3RlLmNvbTASBgNVHRMB
Af8ECDAGAQH/AgEAMDQGA1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudGhhd3Rl
LmNvbS9UaGF3dGVQQ0EuY3JsMA4GA1UdDwEB/wQEAwIBBjAoBgNVHREEITAfpB0w
GzEZMBcGA1UEAxMQVmVyaVNpZ25NUEtJLTItOTAdBgNVHQ4EFgQUp6KDuzRFQD38
1TBPErk+oQGf9tswHwYDVR0jBBgwFoAUe1tFz6/Oy3r9MZIaarbzRutXSFAwDQYJ
KoZIhvcNAQEFBQADggEBAIAigOBsyJUW11cmh/NyNNvGclYnPtOW9i4lkaU+M5en
S+Uv+yV9Lwdh+m+DdExMU3IgpHrPUVFWgYiwbR82LMgrsYiZwf5Eq0hRfNjyRGQq
2HGn+xov+RmNNLIjv8RMVR2OROiqXZrdn/0Dx7okQ40tR0Tb9tiYyLL52u/tKVxp
EvrRI5YPv5wN8nlFUzeaVi/oVxBw9u6JDEmJmsEj9cIqzEHPIqtlbreUgm0vQF9Y
3uuVK6ZyaFIZkSqudZ1OkubK3lTqGKslPOZkpnkfJn1h7X3S5XFV2JMXfBQ4MDzf
huNMrUnjl1nOG5srztxl1Asoa06ERlFE9zMILViXIa4=
-----END CERTIFICATE-----
 2 s:/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2006 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA
   i:/C=ZA/ST=Western Cape/L=Cape Town/O=Thawte Consulting cc/OU=Certification Services Division/CN=Thawte Premium Server <email address hidden>
-----BEGIN CERTIFICATE-----
MIIERTCCA66gAwIBAgIQM2VQCHmtc+IwueAdDX+skTANBgkqhkiG9w0BAQUFADCB
zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ
Q2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UE
CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhh
d3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNl
cnZlckB0aGF3dGUuY29tMB4XDTA2MTExNzAwMDAwMFoXDTIwMTIzMDIzNTk1OVow
gakxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwx0aGF3dGUsIEluYy4xKDAmBgNVBAsT
H0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xODA2BgNVBAsTLyhjKSAy
MDA2IHRoYXd0ZSwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MR8wHQYD
VQQDExZ0aGF3dGUgUHJpbWFyeSBSb290IENBMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEArKDw+4BZ1JzHpM+doVlzCRBFDA0sbmjxbFtIaElZN/wLMxnC
d3/MEC2VNBzm600JpxzSuMmXNgK3idQkXwbAzESUlI0CYm/rWt0RjSiaXISQEHoN
vXRmL2o4oOLVVETrHQefB7pv7un9Tgsp9T6EoAHxnKv4HH6JpOih2HFlDaNRe+68
0iJgDblbnd+6/FFbC6+Ysuku6QToYofeK8jXTsFMZB7dz4dYukpPymgHHRydSsbV
L5HMfHFyHMXAZ+sy/cmSXJTahcCbv1N9Kwn0jJ2RH5dqUsveCTakd9h7h1BE1T5u
KWn7OUkmHgmlgHtALevoJ4XJ/mH9fuZ8lx3VnQIDAQABo4HCMIG/MA8GA1UdEwEB
/wQFMAMBAf8wOwYDVR0gBDQwMjAwBgRVHSAAMCgwJgYIKwYBBQUHAgEWGmh0dHBz
Oi8vd3d3LnRoYXd0ZS5jb20vY3BzMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQU
e1tFz6/Oy3r9MZIaarbzRutXSFAwQAYDVR0fBDkwNzA1oDOgMYYvaHR0cDovL2Ny
bC50aGF3dGUuY29tL1RoYXd0ZVByZW1pdW1TZXJ2ZXJDQS5jcmwwDQYJKoZIhvcN
AQEFBQADgYEAhKhMyT4qvJrizI8LsiV3xGGJiWNa1KMVQNT7Xj+0Q+pjFytrmXSe
Cajd1FYVLnp5MV9jllMbNNkV6k9tcMq+9oKp7dqFd8x2HGqBCiHYQZl/Xi6Cweiq
95OBBaqStB+3msAHF/XLxrRMDtdW3HEgdDjWdMbWj2uvi42gbCkLYeA=
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=Ohio/L=Mentor/O=CardinalCommerce Corporation/CN=*.cardinalcommerce.com
issuer=/C=US/O=Thawte, Inc./CN=Thawte SSL CA
---
No client certificate CA names sent
---
SSL handshake has read 3607 bytes and written 482 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol : SSLv3
    Cipher : RC4-MD5
    Session-ID: C754AC25DD0CDCA9957AB37124377C6E20C152367F5853731119E1145D9891EE
    Session-ID-ctx:
    Master-Key: A8C79125BCDDECAE39481DB3F9B2052152B005A19AEC110535B445555A6EEF0015BECB78A47967616CA22DFB1B824498
    Key-Arg : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1397060780
    Timeout : 7200 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---

``` CURL
curl https://centinel1000.cardinalcommerce.com/maps/txns.asp
curl: (35) Unknown SSL protocol error in connection to centinel1000.cardinalcommerce.com:443

``` CURL with explicit version
# curl -vvv --sslv3 https://centinel1000.cardinalcommerce.com/maps/txns.asp
* Hostname was NOT found in DNS cache
* Trying 216.150.133.226...
* Connected to centinel1000.cardinalcommerce.com (216.150.133.226) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Server hello (2):
* error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
* Closing connection 0
curl: (35) error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

This is in Ubuntu 14.04 Trusty and I have 'apt-get upgrade' and 'update-ca-certificate --fresh'
Note that most/all of these have been tested in MacOSX OpenSSL 0.9.8y 5 Feb 2013 and Ubuntu 12.04 OpenSSL 1.0.1 14 Mar 2012

I will attempt to compile or downgrade myself now.

[Jared Kipe (jared-n)]

Looks like the problem is that 'RC4-MD5' cipher is disabled by default.

I cannot figure out how to enable it by default, but instead just set the curl opt for it and everything is fine.

[Jared Kipe (jared-n)]

EDIT: And by disabled, I mean it doesn't auto-negotiate to it. Wether or not that is 'disabled' or just a bug, it is hard to tell. (I'm no curl or openssl expert for sure)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Alex Muntada (alex.muntada)]

@jared-n This work-around should work:

curl --ciphers RC4-SHA:RC4-MD5 https://...

[Andrew (radamanf)]

I'm affected too, my 2x servers and local PC behave exactly the same.
Alex thank you for this workaround! It's WORKING :)

> curl --ciphers RC4-SHA:RC4-MD5 https://...

[Jared Kipe (jared-n)]

@alex.muntada Yes, as my frist reply mentioned, the problem is missing RC4-MD5 cipher. There are innumerable ways to call into curl as a library, all of which SHOULD have some way to add that cipher. (PHP/HHVM is where I noticed the bug first)

I do not believe this is a bug in curl, as much as poor/aggressive defaults in openssl lib.

[Richard Huffman (rhuffman)]

We're experiencing the same problem, but the fix listed above does not help.

---Initial error:

greatnature-qa:~$ openssl s_client -msg -connect inaturalist.org:443CONNECTED(00000003)
>>> TLS 1.2 Handshake [length 013b], ClientHello
    01 00 01 37 03 03 53 cd 1d 0f 75 28 af 21 9d 17
    62 73 2d 03 70 69 5a d0 27 4d 3f bd f7 bc 55 4f
    e6 76 e7 6f e5 2e 00 00 9e c0 30 c0 2c c0 28 c0
    24 c0 14 c0 0a c0 22 c0 21 00 a3 00 9f 00 6b 00
    6a 00 39 00 38 00 88 00 87 c0 32 c0 2e c0 2a c0
    26 c0 0f c0 05 00 9d 00 3d 00 35 00 84 c0 12 c0
    08 c0 1c c0 1b 00 16 00 13 c0 0d c0 03 00 0a c0
    2f c0 2b c0 27 c0 23 c0 13 c0 09 c0 1f c0 1e 00
    a2 00 9e 00 67 00 40 00 33 00 32 00 9a 00 99 00
    45 00 44 c0 31 c0 2d c0 29 c0 25 c0 0e c0 04 00
    9c 00 3c 00 2f 00 96 00 41 c0 11 c0 07 c0 0c c0
    02 00 05 00 04 00 15 00 12 00 09 00 14 00 11 00
    08 00 06 00 03 00 ff 02 01 00 00 6f 00 0b 00 04
    03 00 01 02 00 0a 00 34 00 32 00 0e 00 0d 00 19
    00 0b 00 0c 00 18 00 09 00 0a 00 16 00 17 00 08
    00 06 00 07 00 14 00 15 00 04 00 05 00 12 00 13
    00 01 00 02 00 03 00 0f 00 10 00 11 00 23 00 00
    00 0d 00 22 00 20 06 01 06 02 06 03 05 01 05 02
    05 03 04 01 04 02 04 03 03 01 03 02 03 03 02 01
    02 02 02 03 01 01 00 0f 00 01 01
139705995765408:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 320 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
greatnature-qa:~$

---Attempted connect with workaround listed above:

greatnature-qa:~$ curl --ciphers RC4-SHA:RC4-MD5 https://inaturalist.org
curl: (35) Unknown SSL protocol error in connection to inaturalist.org:443
greatnature-qa:~$

rhuffman@greatnature-qa:~$ uname -a
Linux greatnature-qa 3.2.0-67-generic #101-Ubuntu SMP Tue Jul 15 17:46:11 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
rhuffman@greatnature-qa:~$

rhuffman@greatnature-qa:~$ cat /etc/debian_version
wheezy/sid
rhuffman@greatnature-qa:~$

MOTD welcome info:
Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.2.0-67-generic x86_64)

[Seth Arnold (seth-arnold)]

Richard, it would be best to open a new bug if you're still experiencing this issue. Thanks!

[Alyssa Rowan (akr)]

Warning: Both RC4 and MD5 are INSECURE. They are susceptible to practical attacks. Do not use them.

MD5 is already disabled by default. Real collisions have been produced, and used to forge certificates in the wild; its use as an HMAC is also strongly discouraged. It must never be used.

RC4 (both RC4-MD5, RC4-SHA and other RC4 ciphers) is a very old stream cipher. It is thought some adversaries can already break it in real-time; in the public literature, several serious weaknesses have already been found (and at the time of writing, another one is on the way). An RFC will shortly be published - see <https://tools.ietf.org/html/draft-popov-tls-prohibiting-rc4-02> - entirely prohibiting the use of all RC4 ciphersuites in all circumstances. Some browsers are already in the process of turning it off.

Please see the results at:
- https://www.ssllabs.com/ssltest/analyze.html?d=centinel1000.cardinalcommerce.com
- https://www.ssllabs.com/ssltest/analyze.html?d=inaturalist.org
which indicate that these sites have deep problems with their encryption.

centinel1000.cardinalcommerce.com seems to be run from an outdated Windows Server 2003 using IIS/6.0 (which hits end-of-life in about a year). It only offers insecure ciphersuites RC4-MD5 & RC4-SHA, and only over SSLv3 (it is intolerant of modern TLS 1.2 connections). You will see from the results that current versions of all mainstream browsers already refuse to connect to it, and in particular I must be clear it is NOT A BUG that curl and wget also refuse to do so - that is correct behaviour and should be regarded as bad as if it offered only 'export' ciphers. Its encryption is exploitably bad: I would consider it in breach of PCI requirements.

inaturalist.com does not support TLS 1.2, uses RC4 (insecure) in preference to other ciphersuites, and offers 1024-bit DHE which is insecure. IE11 does the best it can there and connects with TLS_RSA_WITH_AES_128_CBC_SHA (0x2f); this is susceptible to BEAST, but not as bad as the above. The problem being reported by curl is that inaturalist.com is intolerant of TLS 1.2. This is also NOT A BUG with the client, but is a bug with the server. Some browsers retry with lower protocol versions automatically (and should use the "downgrade" SCSV to indicate this, as this is otherwise behaviour exploitable by an attacker); curl and wget do not.

It is strongly likely that future versions of TLS libraries will completely ignore requests to use these ciphersuites: libReSSL disables it, and I think BoringSSL might too. At best, this is a stop-gap measure, but you should be aware the problem does not lie with you here. I suggest you contact the respective sites' security departments to inform them their encryption is weak.

As this does not seem to be a bug in the client, I suggest closing this one.

[Evgeniy (vdvjekanavi39)]

We have encountered the same error when where using Amazon Cloud EC2 servers. The server that have encountered this error was 14.04 version. The older versions like 12.04 does not have such problem.
Looks like amazon team are building packages on servers with different options.

Hope this will help someone in future.

[Graham Leggett (minfrin-y)]

I've also slammed headlong into this one.

The clue is "SSL handshake has read 0 bytes and written 317 bytes"

What the openssl v1.0.1f client side is doing is sending a clienthello packet larger than 255 bytes to a broken SSL implementation, which slams the phone down on you, thus "read 0 bytes".

The openssl client side errors handling is currently broken, and does not clearly indicate that the connection was dropped, just the vague message that a handshake failure occurred (I've logged this bug here: https://github.com/openssl/openssl/issues/4706)

The suggestion to limit the list of ciphers to just two works around the problem because the clienthello is vastly reduced in size. Obviously this works where your chosen ciphers are accepted by the server, but won't work with the same confusingly identical error message when the ciphers are not supported by the server.

The tangent about MD5 above, while true, has nothing whatsoever to do with this bug.

It looks like the default cipher list on the client side has grown way too long, and when an application offers no control over the cipher list this breaks connections to buggy SSL servers.

Turns out one such buggy SSL server implementation is openssl v1.0.1f as supplied by Ubuntu Xenial, that is covered here: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1612711

As to this client side bug, we need to figure out how to ensure the default cipher list stays well below the 255 byte limit, especially since the SNI header has to fit inside 255 bytes too.

[Adrien Nader (adrien)]

RC4-MD5 was already considered pretty bad when this bug was filled; now they're clearly deprecated and Ubuntu's openssl is actively pushing for higher security standards. As such I think this bug should be WONTFIX.