fopen fails on some SSL urls

[Scott Beardsley]

Binary package hint: php5

Description: Ubuntu 10.04 LTS
Release: 10.04

php5:
  Installed: 5.3.2-1ubuntu4.2
  Candidate: 5.3.2-1ubuntu4.2
  Version table:
 *** 5.3.2-1ubuntu4.2 0
        500 http://archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     5.3.2-1ubuntu4 0
        500 http://archive.ubuntu.com/ubuntu/ lucid/main Packages

For some reason I can't seem to get the following to work. I suspect a SSL problem. Maybe the intermediate SSL cert is not being recognized properly? The server cert is signed by geotrust (which is an intermediate of equifax[1]).

I put the following in a file called /tmp/fopen.php:

<?php
if (fopen("https://www.google.com","r")) { print "www.google.com worked\n"; }
if (fopen("https://cas.ucdavis.edu","r")) { print "cas.ucdavis.edu worked\n"; }
?>

Then I run the php via an apache web and/or via the php5-cli (the results are the same in both cases):

$ php /tmp/fopen.php
www.google.com worked
PHP Warning: fopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:140773F2:SSL routines:func(119):reason(1010) in /tmp/fopen.php on line 3
PHP Warning: fopen(): Failed to enable crypto in /tmp/fopen.php on line 3
PHP Warning: fopen(https://cas.ucdavis.edu): failed to open stream: operation failed in /tmp/fopen.php on line 3
$

When I run the above command on a karmic or jaunty machine it works fine for both fopen() calls. I've attached a tcpdump of the above script.

As you can see from the dump, Google is working but my server is not. I get an SSL alert packet (packet #29) back with code 10
(unexpected message). Maybe this is an intermediate cert verification problem?

What is funny is that I get an ACK right before that. It seems like maybe the server is sending an ACK, client starts talking, server isn't ready and sends an out-of-order message.

Scott
-----------
[1] https://www.geotrust.com/resources/root-certificates/index.html

[Clint Byrum (clint-fewbar)]

The example given returns the same result for me on an up to date maverick system. I think the problem is just a misleading error message bubbling up from openssl. s_client does give an error about the self signed cert:

verify error:num=19:self signed certificate in certificate chain

Full log:

clint@ubuntu:~$ openssl s_client -host cas.ucdavis.edu -port 443
CONNECTED(00000003)
depth=1 /C=US/O=Equifax/OU=Equifax Secure Certificate Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=Davis/O=University of California Davis/OU=IET-IR/CN=cas.ucdavis.edu
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
 1 s:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC/DCCAmWgAwIBAgIDCiCtMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT
MRAwDgYDVQQKEwdFcXVpZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkwHhcNMDgxMTA2MjMwNDQ2WhcNMTEwMTA2MjMwNDQ2
WjCBhjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExDjAMBgNVBAcT
BURhdmlzMScwJQYDVQQKEx5Vbml2ZXJzaXR5IG9mIENhbGlmb3JuaWEgRGF2aXMx
DzANBgNVBAsTBklFVC1JUjEYMBYGA1UEAxMPY2FzLnVjZGF2aXMuZWR1MIGfMA0G
CSqGSIb3DQEBAQUAA4GNADCBiQKBgQDRT3t20tSOMW9sC+WYk8csHzV6JK+aMGd8
m9NDQtK3bb5STyp1AfuovU2tGKv1YD5HCIs1BzDbbN+XJIrU+zSAdrVdHKp62ZKy
AWTFfwfQ0VWvBz8iKzWVpfiRutUC+RqodMBQ3DqM0YU4RX6cz9L5QFi+hQsCQ+Ha
lKzseuEJnQIDAQABo4GuMIGrMA4GA1UdDwEB/wQEAwIE8DAdBgNVHQ4EFgQUZoEl
UbQzpXvJyk5JVUGmVQu5Ka0wOgYDVR0fBDMwMTAvoC2gK4YpaHR0cDovL2NybC5n
ZW90cnVzdC5jb20vY3Jscy9zZWN1cmVjYS5jcmwwHwYDVR0jBBgwFoAUSOZo+SvS
spXXR9gjIBBPM5iQn9QwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMA0G
CSqGSIb3DQEBBQUAA4GBADhAlAHFmemcwilbfWfu2//Os58jzJNCBFPNpS0d+tg4
AQTgR4Ogs7ljbJeo4+2eEnGvLHvPy1El8JkKRexwVhQSymz60Bnkg0oiQ6qIYwML
r5Gfk+liSBpexjZkPp+olFO8u/d+UlW6ZPfI5RTyz5e+InrETFyjgoIJY3y3SnFQ
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Davis/O=University of California Davis/OU=IET-IR/CN=cas.ucdavis.edu
issuer=/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
No client certificate CA names sent
---
SSL handshake has read 2147 bytes and written 276 bytes
---
New, TLSv1/SSLv3, Cipher is EDH-RSA-DES-CBC3-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol : TLSv1
    Cipher : EDH-RSA-DES-CBC3-SHA
    Session-ID: 4C116AFE454ACEE059BF6889329DDEB55963208CB0353EBCB8F2774B3B1A92A5
    Session-ID-ctx:
    Master-Key: B7D3BB1CA375E594F0E82EE8EB4CD3FAD33B17E96BFFCD34DDF95AA02EBE439C2ED9E0216F96E2205E35237610A50869
    Key-Arg : None
    Start Time: 1276209918
    Timeout : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---
HEAD / HTTP/1.0
Host: cas.ucdavis.edu

HTTP/1.1 302 Moved Temporarily
Server: Apache-Coyote/1.1
Location: https://cas.ucdavis.edu/login
Content-Type: text/html;charset=ISO-8859-1
Content-Length: 0
Date: Thu, 10 Jun 2010 22:45:34 GMT
Connection: close

closed
clint@ubuntu:~$

[Scott Beardsley (sc0ttbeardsley)]

I noticed that too. The necessary CAs are actually installed on Lucid by default though (you just have to tell openssl where to look). Incidentally, wget works fine (without --no-check-certificate):

$ openssl s_client -CApath /etc/ssl/certs -connect cas.ucdavis.edu:443
CONNECTED(00000003)
depth=1 /C=US/O=Equifax/OU=Equifax Secure Certificate Authority
verify return:1
depth=0 /C=US/ST=California/L=Davis/O=University of California Davis/OU=IET-IR/CN=cas.ucdavis.edu
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Davis/O=University of California Davis/OU=IET-IR/CN=cas.ucdavis.edu
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
 1 s:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC/DCCAmWgAwIBAgIDCiCtMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT
MRAwDgYDVQQKEwdFcXVpZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkwHhcNMDgxMTA2MjMwNDQ2WhcNMTEwMTA2MjMwNDQ2
WjCBhjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExDjAMBgNVBAcT
BURhdmlzMScwJQYDVQQKEx5Vbml2ZXJzaXR5IG9mIENhbGlmb3JuaWEgRGF2aXMx
DzANBgNVBAsTBklFVC1JUjEYMBYGA1UEAxMPY2FzLnVjZGF2aXMuZWR1MIGfMA0G
CSqGSIb3DQEBAQUAA4GNADCBiQKBgQDRT3t20tSOMW9sC+WYk8csHzV6JK+aMGd8
m9NDQtK3bb5STyp1AfuovU2tGKv1YD5HCIs1BzDbbN+XJIrU+zSAdrVdHKp62ZKy
AWTFfwfQ0VWvBz8iKzWVpfiRutUC+RqodMBQ3DqM0YU4RX6cz9L5QFi+hQsCQ+Ha
lKzseuEJnQIDAQABo4GuMIGrMA4GA1UdDwEB/wQEAwIE8DAdBgNVHQ4EFgQUZoEl
UbQzpXvJyk5JVUGmVQu5Ka0wOgYDVR0fBDMwMTAvoC2gK4YpaHR0cDovL2NybC5n
ZW90cnVzdC5jb20vY3Jscy9zZWN1cmVjYS5jcmwwHwYDVR0jBBgwFoAUSOZo+SvS
spXXR9gjIBBPM5iQn9QwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMA0G
CSqGSIb3DQEBBQUAA4GBADhAlAHFmemcwilbfWfu2//Os58jzJNCBFPNpS0d+tg4
AQTgR4Ogs7ljbJeo4+2eEnGvLHvPy1El8JkKRexwVhQSymz60Bnkg0oiQ6qIYwML
r5Gfk+liSBpexjZkPp+olFO8u/d+UlW6ZPfI5RTyz5e+InrETFyjgoIJY3y3SnFQ
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Davis/O=University of California Davis/OU=IET-IR/CN=cas.ucdavis.edu
issuer=/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
No client certificate CA names sent
---
SSL handshake has read 2147 bytes and written 276 bytes
---
New, TLSv1/SSLv3, Cipher is EDH-RSA-DES-CBC3-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol : TLSv1
    Cipher : EDH-RSA-DES-CBC3-SHA
    Session-ID: 4C116E5221F8596C7B1BE3E4443D427A6234FCE19A12F6E869C3F0C536715A7D
    Session-ID-ctx:
    Master-Key: C52784FE43D5156FDB3A81670E1BF87585502BC5C38EAE214F2C93285743BB8B050B8B111751A7B16A3784159B6444B3
    Key-Arg : None
    Start Time: 1276210770
    Timeout : 300 (sec)
    Verify return code: 0 (ok)
---
HEAD / HTTP/1.0

HTTP/1.1 302 Moved Temporarily
Server: Apache-Coyote/1.1
Location: https://casweb3.ucdavis.edu:8443/login
Content-Type: text/html;charset=ISO-8859-1
Content-Length: 0
Date: Thu, 10 Jun 2010 22:59:33 GMT
Connection: close

closed
$ wget https://cas.ucdavis.edu
--2010-06-10 16:01:53-- https://cas.ucdavis.edu/
Resolving cas.ucdavis.edu... 169.237.104.82
Connecting to cas.ucdavis.edu|169.237.104.82|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://cas.ucdav...

[Clint Byrum (clint-fewbar)]

Hmm furthermore, I don't think this is a certificate verification problem, because the manual for the SSL wrapper (which https urls ride on top of) states that verify_peer defaults to false. If you run this code with 'strace -e trace=open,stat php xxxx.php' you can see that the ssl certs dir doesn't even get opened until the google request, suggesting that we never get a cert:

<?php
$ctx = stream_context_create(array('ssl'=>array('verify_peer'=>true, 'capath'=>'/etc/ssl/certs')));
print "now cas.ucdavis.edu...\n";
fopen("https://cas.ucdavis.edu/",'r',false,$ctx);
print "try ssl to google...\n";
fopen("https://www.google.com/",'r',false,$ctx);

[Scott Beardsley (sc0ttbeardsley)]

This is strange... on lucid it doesn't even attempt to check for the CA file. Using the script you provided on a Karmic machine I get the following:

now cas.ucdavis.edu...
open("/etc/host.conf", O_RDONLY) = 3
open("/etc/resolv.conf", O_RDONLY) = 3
open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/lib/libnss_mdns4_minimal.so.2", O_RDONLY) = 3
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libnss_dns.so.2", O_RDONLY) = 3
open("/etc/resolv.conf", O_RDONLY) = 3
open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 4
open("/etc/ssl/certs/594f1775.0", O_RDONLY|O_LARGEFILE) = 4
open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
open("/etc/ssl/certs/594f1775.0", O_RDONLY|O_LARGEFILE) = 4
try ssl to google...
open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
open("/etc/gai.conf", O_RDONLY) = 3
open("/etc/ssl/certs/7651b327.0", O_RDONLY|O_LARGEFILE) = 4
open("/dev/urandom", O_RDONLY) = 0
open("/dev/urandom", O_RDONLY) = 0
open("/dev/urandom", O_RDONLY) = 0

I'm puzzled why Lucid doesn't check for the CA. As you can see from above the server's cert is offered and verified on a Karmic machine. The file referenced above (/etc/ssl/certs/594f1775.0) exists on both machines and has the same sha1sum:

03de306e6bead81b0de390a2c47ba264139e4e69 /etc/ssl/certs/594f1775.0

Long shot, but, I did notice that the Issuer CN on the cas.ucdavis.edu cert doesn't have a value. Is it required?

[Clint Byrum (clint-fewbar)]

It would appear this problem is somewhere upstream. I built the latest vanilla snapshot of 5.3 on both karmic and maverick and the issue still persists:

clint@ubuntu:~/pkg/php5/src/php5.3-201006151430$ sapi/cli/php ../../test_592442.php
now cas.ucdavis.edu...

Warning: fopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:140773F2:SSL routines:func(119):reason(1010) in /home/clint/pkg/php5/test_592442.php on line 4

Warning: fopen(): Failed to enable crypto in /home/clint/pkg/php5/test_592442.php on line 4

Warning: fopen(https://cas.ucdavis.edu/): failed to open stream: operation failed in /home/clint/pkg/php5/test_592442.php on line 4
try ssl to google...
clint@ubuntu:~/pkg/php5/src/php5.3-201006151430$

We should report this on PHP's bugtracker, but I think we need something more repeatable than your server URL.

Can you setup a similarly configured server elsewhere, repeat it, and share the configuration here so we can determine what the issue is exactly?

[C de-Avillez (hggdh2)]

Marking Medium importance per Clint.

[Scott Beardsley (sc0ttbeardsley)]

I've filed a bug report at bugs.php.net[1] and linked to this bug report.

I'll ask for more details about this specific server. I tried a bunch of other sites with a similar setup but can't reproduce it elsewhere yet. Maybe there is a firewall rule that is blocking packets? Seems strange that it would work for php 5.2.10 but not 5.3 though. Hopefully the php folks can provide some insight as to what changed between those versions. Nothing obvious (to me at least) jumps out on the changelog[2]:

Fixed bug #50832 (HTTP fopen wrapper does not support passwordless HTTP authentication). (Jani)
Fixed bug #50791 (Compile failure: Bad logic in defining fopencookie emulation). (Jani)
Fixed bug #48637 ("file" fopen wrapper is overwritten when using --with-curlwrappers). (Jani)
Fixed bug #43510 (stream_get_meta_data() does not return same mode as used in fopen). (Jani)
Optimized require_once() and include_once() by eliminating fopen(3) on second usage. (Dmitry)
Added 'n' flag to fopen to allow passing O_NONBLOCK to the underlying open(2) system call. (Mikko)
Added "ignore_errors" option to http fopen wrapper. (David Zulke, Sara)

Scott
--------
[1] http://bugs.php.net/bug.php?id=52106
[2] http://php.net/ChangeLog-5.php

[Scott Beardsley (sc0ttbeardsley)]

An update... I've discovered that the cas.ucdavis.edu machine does not correctly deal with fragmented IP packets. I thought this might be the problem but then I went looking for those types of packets and found none (I'll still try to get that fixed though).

Then I discovered that on the "client hello" packet (first packet after the connection is established) on a Lucid machine shows up in wireshark as the TLSv1 protocol and on a Karmic machine it shows up as a SSLv2 protocol.

I wonder why this changed? Did the defaults for openssl change or something?

[Clint Byrum (clint-fewbar)]

Scott, thanks for continuing to look into this.

The packet information seems like it would be good information for the PHP bug.

Seeing as I was able to make it fail on karmic w/ the nightly PHP snapshot, its not really "karmic vs. lucid" but rather "PHP 5.2.10 vs. PHP 5.3.2".

[Arjan van der Veen (arjanvanderveen)]

I want to add a comment that the bug is available for more servers:

<?php
#$ufurl = "https://server.db.kvk.nl/"; # FAILS
#$ufurl = "https://gmail.com/"; #OK
#$ufurl = "https://cas.ucdavis.edu/login"; #FAILS
$fp = fopen($ufurl, 'r');
$data = fread($fp, 10000);
var_dump($data);

?>

output:

PHP Warning: fopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:1407741A:SSL routines:func(119):reason(1050) in /tmp/urltest.php on line 7
PHP Warning: fopen(): Failed to enable crypto in /tmp/urltest.php on line 7
PHP Warning: fopen(https://server.db.kvk.nl/): failed to open stream: operation failed in /tmp/urltest.php on line 7
PHP Warning: fread() expects parameter 1 to be resource, boolean given in /tmp/urltest.php on line 8
bool(false)

Does anybody have a workaround?

[Scott Beardsley (sc0ttbeardsley)]

Arjan,

This is an upstream bug. Please leave a comment here[1]. There has been no progress (nor ack's) from the php team regarding this bug.

Scott
----------
[1] http://bugs.php.net/bug.php?id=52106

[Clint Byrum (clint-fewbar)]

So initial testing shows that this is actually a problem with OpenSSL, or at least, it is OpenSSL refusing to connect to these servers:

(natty-amd64)root@clint-MacBookPro:/home/clint/pkg/php5/bzr/natty-php-ssl-fix# openssl s_client -host cas.ucdavis.edu -port 443
CONNECTED(00000003)
1787:error:140773F2:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message:s23_clnt.c:602:
(natty-amd64)root@clint-MacBookPro:/home/clint/pkg/php5/bzr/natty-php-ssl-fix# openssl s_client -host server.db.kvk.nl -port 443
CONNECTED(00000003)
1788:error:1407741A:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert decode error:s23_clnt.c:602:

The condition to reach that error is:

    else if ((p[0] == SSL3_RT_ALERT) &&
         (p[1] == SSL3_VERSION_MAJOR) &&
         ((p[2] == SSL3_VERSION_MINOR) ||
          (p[2] == TLS1_VERSION_MINOR)) &&
         (p[3] == 0) &&
         (p[4] == 2))

So it seems if the SSL version is 3, or we're using tlsv1, the code is triggered (602 is later).

Still investigating, but I am adding an OpenSSL bug task.

[Steve Beattie (sbeattie)]

On Wed, Dec 01, 2010 at 01:25:37AM -0000, Clint Byrum wrote:
> So initial testing shows that this is actually a problem with OpenSSL,
> or at least, it is OpenSSL refusing to connect to these servers:
>
> (natty-amd64)root@clint-MacBookPro:/home/clint/pkg/php5/bzr/natty-php-ssl-fix# openssl s_client -host cas.ucdavis.edu -port 443
> CONNECTED(00000003)
> 1787:error:140773F2:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message:s23_clnt.c:602:
> (natty-amd64)root@clint-MacBookPro:/home/clint/pkg/php5/bzr/natty-php-ssl-fix# openssl s_client -host server.db.kvk.nl -port 443
> CONNECTED(00000003)
> 1788:error:1407741A:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert decode error:s23_clnt.c:602:

Note that if you force openssl to use ssl3 via -ssl3, a successful
connection is made. However, both warn of a self-signed certificate in
the chain, though it appears to be the top level certificate:

$ openssl s_client -host server.db.kvk.nl -port 443 -ssl3
CONNECTED(00000003)
depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
 0 s:/C=NL/ST=Utrecht/L=Woerden/O=Kamer van Koophandel Nederland/OU=Technisch Beheer/CN=SERVER.DB.KVK.NL
   i:/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
 1 s:/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
 2 s:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority

(server.db.kvk.nl's certificate is signed by Equifax, not Verisign.)

But perhaps the self-signed certificate thing is a red-herring, as on
hardy (0.9.8g-4ubuntu3.12) and lucid (0.9.8k-7ubuntu8.4), at least,
connecting works, but still gives the warning.

--
Steve Beattie
<email address hidden>
http://NxNW.org/~steve/

[Steve Beattie (sbeattie)]

Okay, as pointed out in an earlier comment, the self-signed certificate bit is a red-herring.

The failure on maverick looks like it's somehow related to how openssl is attempting to negotiate RFC4507bis session tickets, as running openssl s_client with -no_ticket also works; e.g.: openssl s_client -CApath /etc/ssl/certs -host server.db.kvk.nl -port 443 -no_ticket

[Clint Byrum (clint-fewbar)]

Steve, thanks for looking into this further.

I've unassigned myself from the SSL portion of this, as I am not really the best person to address the issue fully.

According to this:

http://www.openssl.org/news/changelog.html

I see where rfc4507bis support was added in 0.9.8e, then (confusingly) again in 0.9.8n. Maybe it was removed from f or g??

Anyway, the answer to this question suggests that it may be that the server is actually in err here, which maybe OpenSSL should handle more gracefully.

http://stackoverflow.com/questions/2667514/openssl-sessionticket-tls-extension-problem

Seems to me that there is a need then to allow disabling the SessionTicket extension to the SSL context options:

http://us2.php.net/manual/en/context.ssl.php

So, I've submitted this PHP bug which would allow disabling the SessionTicket extension.

http://bugs.php.net/bug.php?id=53447

Marking the PHP portion of this Triaged. I suspect that OpenSSL is actually doing the right thing here, and so suggest that it might be Invalid (though I'd like somebody more familiar with OpenSSL to make that change)

[Launchpad Janitor (janitor)]

This bug was fixed in the package php5 - 5.3.5-1ubuntu6

---------------
php5 (5.3.5-1ubuntu6) natty; urgency=low

  * debian/patches/fpm-config.patch: Update php-fpm.conf(pool.d/con)
    to do initial chdir to / as suggest by Olaf van van der Spek
    to detect early problems if php5-fpm needs a write access to
    initial chdir.
  * debian/patches/backport-upstream-lp592442.patch: Backport upstream fix
    for ssl fopen issues. (LP: #592442)
 -- Chuck Short <email address hidden> Fri, 01 Apr 2011 09:29:49 -0400

[Matt Gallagher (mattgal)]

I'm having a Python + SSL problem for certain hosts and I think they are related. Everything was working fine on 10.04 then I just recently upgraded to 10.10 and certain https sites don't load using python's httplib.

Booting off an 11.04 beta 1 livecd:

$ uname -a
Linux ubuntu 2.6.38-7-generic #39-Ubuntu SMP Fri Mar 25 21:24:57 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

$ openssl s_client -host bpayview.salmat.com.au -port 443
CONNECTED(00000003)
5409:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:

$ openssl s_client -host www.pennytel.com -port 443
CONNECTED(00000003)
5411:error:140773F2:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message:s23_clnt.c:602:

$ dpkg --list | grep -i ssl
ii libssl0.9.8 0.9.8o-5ubuntu1 SSL shared libraries
ii openssl 0.9.8o-5ubuntu1 Secure Socket Layer (SSL) binary and related cryptographic tools
ii python-openssl 0.10-1ubuntu2 Python wrapper around the OpenSSL library
ii ssl-cert 1.0.28 simple debconf wrapper for OpenSSL

[Finjon Kiang (kiange)]

It looked like this bug is still there.

I used the code below to test:
<?php file_get_contents('https://aquarius.neweb.com.tw');

Three environments:
# PHP Version 5.3.5-1ubuntu7.3
Suhosin Patch 0.9.10
Apache/2.2.17 (Ubuntu)
OpenSSL 0.9.8o 01 Jun 2010

# PHP Version 5.3.6-13ubuntu3.2
Suhosin Patch 0.9.10
Apache/2.2.20 (Ubuntu)
OpenSSL 1.0.0e 6 Sep 2011

# PHP Version 5.2.4-2ubuntu5.17
Suhosin Patch 0.9.6.2
Apache/2.2.8 (Ubuntu)
OpenSSL 0.9.8g 19 Oct 2007

Only the one of PHP Version 5.2.4 could get correct response. The other two returned the errors like:
Warning: file_get_contents() [function.file-get-contents]: Failed to enable crypto in xxx.php on line 2
Warning: file_get_contents(https://aquarius.neweb.com.tw) [function.file-get-contents]: failed to open stream: operation failed in xxx.php on line 2

I originally guess it's because the domain I was trying to communicate, https://aquarius.neweb.com.tw, is using Wildcard SSL Certificate (*.neweb.com.tw). But there's no error when communicating with https://encrypted.google.com/ , which is using *.google.com.

[Finjon Kiang (kiange)]

Three environments:
# PHP Version 5.3.5-1ubuntu7.3
Suhosin Patch 0.9.10
Apache/2.2.17 (Ubuntu)
OpenSSL 0.9.8o 01 Jun 2010

$ uname -a
Linux xxx #50-Ubuntu SMP Mon Sep 12 21:51:23 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
$ openssl s_client -host aquarius.neweb.com.tw -port 443
CONNECTED(00000003)
28269:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:

---

# PHP Version 5.3.6-13ubuntu3.2
Suhosin Patch 0.9.10
Apache/2.2.20 (Ubuntu)
OpenSSL 1.0.0e 6 Sep 2011

$ uname -a
Linux xxx 3.0.0-13-generic #22-Ubuntu SMP Wed Nov 2 13:27:26 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
$ openssl s_client -host aquarius.neweb.com.tw -port 443
CONNECTED(00000003)
140055608010400:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:591:

---

# PHP Version 5.2.4-2ubuntu5.17
Suhosin Patch 0.9.6.2
Apache/2.2.8 (Ubuntu)
OpenSSL 0.9.8g 19 Oct 2007

$ uname -a
Linux xxx 2.6.24-17-server #1 SMP Thu May 1 14:28:06 UTC 2008 x86_64 GNU/Linux
$ openssl s_client -host aquarius.neweb.com.tw -port 443
CONNECTED(00000003)
depth=2 /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
verify error:num=19:self signed certificate in certificate chain
verify return:0

[Clint Byrum (clint-fewbar)]

Excerpts from Finjon Kiang's message of Fri Dec 02 15:28:56 UTC 2011:
> Three environments:
> # PHP Version 5.3.5-1ubuntu7.3
> Suhosin Patch 0.9.10
> Apache/2.2.17 (Ubuntu)
> OpenSSL 0.9.8o 01 Jun 2010
>
> $ uname -a
> Linux xxx #50-Ubuntu SMP Mon Sep 12 21:51:23 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> $ openssl s_client -host aquarius.neweb.com.tw -port 443
> CONNECTED(00000003)
> 28269:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:
>

The site doesn't seem to support SSLv3

$ openssl s_client -host aquarius.neweb.com.tw -port 443
CONNECTED(00000003)
140489793156768:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:591:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol : SSLv3
    Cipher : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg : None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1322848731
    Timeout : 7200 (sec)
    Verify return code: 0 (ok)
---

On an older openssl:

$ openssl s_client -host aquarius.neweb.com.tw -port 443
CONNECTED(00000003)
depth=2 /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
verify return:1
depth=1 /C=US/ST=UT/L=Salt Lake City/O=The USERTRUST Network/OU=http://www.usertrust.com/CN=UTN-USERFirst-Hardware
verify return:1
depth=0 /C=TW/postalCode=11510/ST=Taiwan/L=Taipei/streetAddress=7F., No.52, Sec. 3, Nangang Rd., Nangang Dist., Taipei City 11510, Taiwan (R.O.C.)/O=Neweb Technologies Co., Ltd./OU=MIS/OU=Provided by Global Digital Inc./OU=GlobalTrustSSLWildcard/CN=*.neweb.com.tw
verify return:1
---
Certificate chain
 0 s:/C=TW/postalCode=11510/ST=Taiwan/L=Taipei/streetAddress=7F., No.52, Sec. 3, Nangang Rd., Nangang Dist., Taipei City 11510, Taiwan (R.O.C.)/O=Neweb Technologies Co., Ltd./OU=MIS/OU=Provided by Global Digital Inc./OU=GlobalTrustSSLWildcard/CN=*.neweb.com.tw
   i:/C=US/ST=UT/L=Salt Lake City/O=The USERTRUST Network/OU=http://www.usertrust.com/CN=UTN-USERFirst-Hardware
 1 s:/C=US/ST=UT/L=Salt Lake City/O=The USERTRUST Network/OU=http://www.usertrust.com/CN=UTN-USERFirst-Hardware
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
 2 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGPjCCBSagAwIBAgIRAPzAoe3QmtGF36gPqlNpvCwwDQYJKoZIhvcNAQEFBQAw
gZcxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJVVDEXMBUGA1UEBxMOU2FsdCBMYWtl
IENpdHkxHjAcBgNVBAoTFVRoZSBVU0VSVFJVU1QgTmV0d29yazEhMB8GA1UECxMY
aHR0cDovL3d3dy51c2VydHJ1c3QuY29tMR8wHQYDVQQDExZVVE4tVVNFUkZpcnN0
LUhhcmR3YXJlMB4XDTExMDExNDAwMDAwMFoXDTE0MDExMzIzNTk1OVowggE1MQsw
CQYDVQQGEwJUVzEOMAwGA1UEERMFMTE1MTAxDzANBgNVBAgTBlRhaXdhbjEPMA0G
A1UEBxMGVGFpcGVpMVswWQYDVQQJE1I3Ri4sIE5vLjUyLCBTZWMuIDMsIE5hbmdh
bmcgUmQuLCBOYW5nYW5nIERpc3QuLCBUYWlwZWkgQ2l0eSAxMTUxMCwgVGFpd2Fu
IChSLk8uQy4pMSUwIwYDVQQKExxOZXdlYiBUZWNobm9sb2dp...

[Finjon Kiang (kiange)]

@Clint, thanks for the test. I've created another bug

https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/899464

[W. Scott Howard (wscotthoward)]

Using

PHP 5.3.6-13ubuntu3.2 with Suhosin-Patch (cli) (built: Oct 13 2011 23:19:13)
Copyright (c) 1997-2011 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies

I can connect via cli

$openssl s_client -host www.gvmax.com -port 443

however PHP gives me the

PHP Warning: file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:
error:140773F2:SSL routines:func(119):reason(1010)

Error

[Anders Østhus (aosthus)]

I'm seeing this issue on 12.04 as well.

Just for completeness I've tested with the same script on the latest 10.04, 10.10, 11.04, 11.10 and 12.04.

The script I'm using is:
<?php
$ufurl = "https://graph.facebook.com/19292868552";
$fp = fopen($ufurl, 'r');
$data = fread($fp, 10000);
var_dump($data);
?>

Results:
10.04:
PHP 5.3.2-1ubuntu4.14 with Suhosin-Patch
OpenSSL 0.9.8k 25 Mar 2009
Data recieved from Facebook.

10.10:
PHP 5.3.3-1ubuntu9.10 with Suhosin-Patch
OpenSSL 0.9.8o 01 Jun 2010
Data recieved from Facebook.

11.04:
PHP 5.3.5-1ubuntu7.7 with Suhosin-Patch
OpenSSL 0.9.8o 01 Jun 2010
Data recieved from Facebook.

11.10:
PHP 5.3.6-13ubuntu3.6 with Suhosin-Patch
OpenSSL 1.0.0e 6 Sep 2011
Data recieved from Facebook.

12.04 (latest available packages):
PHP 5.3.10-1ubuntu3 with Suhosin-Patch
OpenSSL 1.0.1 14 Mar 2012
Result:
PHP Warning: fopen(): SSL: crypto enabling timeout in /home/ubuntu/ssltest.php on line 3
PHP Warning: fopen(): Failed to enable crypto in /home/ubuntu/ssltest.php on line 3
PHP Warning: fopen(https://graph.facebook.com/19292868552): failed to open stream: operation failed in /home/ubuntu/ssltest.php on line 3
PHP Warning: fread() expects parameter 1 to be resource, boolean given in /home/ubuntu/ssltest.php on line 4
bool(false)

So it seems like this is a regression in either PHP or OpenSSL. I'm guessing OpenSSL, since I'm seeing similar behavior in Ruby aswell.

[Anders Østhus (aosthus)]

Seems like this issue is tracked in #965371 for Precise.

[Clint Byrum (clint-fewbar)]

Anders, I'm not entirely sure that is the same issue. I've tried an affected openssl s_client on a few of the other noted servers from other comments on the bug, and they are not failing. That said, the original problem with the ucdavis server does not happen anymore for me on Ubuntu 12.04. I think the real problem was PHP not handling the error at all.

[chrone (chrone81)]

i'm having the same problem here after upgrade from 11.10, my web server could not set email using curl and google mail smtp.

i guess the culprit is either between php5-curl, curl, and openssl. :(

is there a way to downgrade each curl and openssl version but still running on ubuntu 12.04 until this bug is fixed?

here's the apache error log:
PHP Warning: file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:\nerror:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure in /var/www/piwik.php on line 114
PHP Warning: file_get_contents(): Failed to enable crypto in /var/www/piwik.php on line 114

[chrone (chrone81)]

downgraded to openssl 1.0.0e-2ubuntu4.5 didn't solve the problem too. :(

[Clint Byrum (clint-fewbar)]

Excerpts from chrone's message of Mon Apr 30 09:34:18 UTC 2012:
> i'm having the same problem here after upgrade from 11.10, my web server
> could not set email using curl and google mail smtp.
>
> i guess the culprit is either between php5-curl, curl, and openssl. :(
>
> is there a way to downgrade each curl and openssl version but still
> running on ubuntu 12.04 until this bug is fixed?
>
> here's the apache error log:
> PHP Warning: file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:\nerror:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure in /var/www/piwik.php on line 114
> PHP Warning: file_get_contents(): Failed to enable crypto in /var/www/piwik.php on line 114
>

You may want to try setting the cipher to use, as the issue seems to be
with a too-large header for some servers to handle.

http://php.net/manual/en/context.ssl.php

You can test what ciphers work with:

openssl s_client -connect server:port -cipher xxxxx

I'd recommend 'AES256' or 'AES128'

[Adrien Nader (adrien)]

It looks like php5 was changed to accomodate whatever openssl was doing. It's difficult to tell whether something has been changed on the openssl side in the meantime but considering how long it's been, I see no reason to keep this bug open.