libwww-curl-perl is compiled without libssl, which breaks functionality

[Patola]

Binary package hint: libwww-curl-perl

It might be related to debian bug #520595.

libwww-curl-perl 4.05-1 was compiled with openssl libs, libwww-curl-perl 4.07-1 is compiled without it. That breaks functionality: a program I developed using perl no longer works with an HTTP certificate. When I revert back to 4.05-1 it starts working again.

The differences from 4.05 Curl.so "ldd" command to 4.07's are:

4.05 shows:
< libcurl.so.4
< libssl.so.0.9.8
< libcrypto.so.0.9.8

4.07 shows:
> libcurl-gnutls.so.4

This is actual output from my perl program using libwww-curl-perl 4.07:

*** LOGIN: We'll log in to Maximo.
*** URL: https://xxxxxxx.xxx.xxx.com/sdm/j_security_check
*** POST: j_username=myusername&j_password=mypassword&login=true&langcode=EN&login_hidden.x=0&login_hidden.y=0
**** ONLY_POST: url=https://xxxxxxx.xxx.xxx.com/sdm/j_security_check, post=j_username=myusername&j_password=mypassword&login=true&langcode=EN&login_hidden.x=0&login_hidden.y=0
* About to connect() to xxxxxxx.xxx.xxx.com port 443 (#0)
* Trying 200.200.200.200... * connected
* Connected to xxxxxxx.xxx.xxx.com (200.200.200.200) port 443 (#0)
* found 145 certificates in /etc/ssl/certs/ca-certificates.crt
* gnutls_handshake() failed: Decryption has failed.
* Closing connection #0
*** ERROR When try to use url: https://xxxxxxx.xxx.xxx.com/sdm/j_security_check, post j_username=myusername&j_password=mypassword&login=true&langcode=EN&login_hidden.x=0&login_hidden.y=0 -- Resource temporarily unavailable, 35

----------------------------

When it worked (4.05-1), it went like that:

*** LOGIN: We'll log in to Maximo.
*** URL: https://xxxxxx.xxx.xxx.com/sdm/j_security_check
*** POST: j_username=myusername&j_password=mypassword&login=true&langcode=EN&login_hidden.x=0&login_hidden.y=0
**** ONLY_POST: url=https://ixxxxxx.xxx.xxx.com/sdm/j_security_check, post=j_username=myusername&j_password=mypassword&login=true&langcode=EN&login_hidden.x=0&login_hidden.y=0
* About to connect() to xxxxxx.xxx.xxx.com port 443 (#0)
* Trying 200.200.200.200... * connected
* Connected to xxxxxx.xxx.xxx.com (200.200.200.200) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
  CApath: /etc/ssl/certs
* SSL connection using AES256-SHA
* Server certificate:
* subject: C=US, ST=New York, L=Poughkeepsie, O=IBM, CN=xxxxxx.xxx.xxx.com
* start date: 2009-10-04 17:17:20 GMT
* expire date: 2012-10-05 22:53:18 GMT
* issuer: C=US, O=Equifax, OU=Equifax Secure Certificate Authority
* SSL certificate verify ok.
> POST /sdm/j_security_check HTTP/1.1
User-Agent: Mozilla/5.0 (compatible; MSIE 6.01; Windows NT 5.0)
Host: xxxxxx.xxx.xxx.com
Accept: */*
Content-Length: 107
Content-Type: application/x-www-form-urlencoded

< HTTP/1.1 302 Found
< Date: Tue, 29 Dec 2009 16:49:52 GMT
< Server: IBM_HTTP_Server/6.0.2.33 Apache/2.0.47 (Unix)
< Location: https://xxxxxx.xxx.xxx.com/sdm/
* Added cookie LtpaToken2="kfaNDGliRobAc8VL15GCnLFlr2PbXOQpkQWnO3UEBlieCQ9ShCa5db753IrowtC2JDkAaAsmSiQi0W4SdYAx1JAfA5DZgk2/Qa76xyIq8TibMe4ik0Is3xf0aVOPC62JNKqxGf56D8b3UPFiYFLmEVAgOlkMCE96JuWiG+6F4BVbih2qxFdAtipZ0pFMU5x4nFyOnKEiL6fcSiEh7Tj0JqMQoYzi2re12Ci3Y0HDUThnx+C69pczG4QrqVsbqZOfvZGgEczaFfNlV22Q7W8sH9upNifxodt74zmHbKPE7crowywTIqmtTQLaEUBzb8Ypa2H5+5u/CGjaf0KlLwzTnq+DGxSQD4PH6DkKJ5xI51rlDseFTpklx1zkAx6pEv3GlLofJnHzpFydW98F3sVZeYPWCWSPZffZbKf2NsfJDZfpf7+rRrOJxB8KeEMC3kjid9Qs+zn7E4p7eyBjzIR2J51Zx8qnBFGbv+NcdOZodTpzfQ7kg21sxzviE/qt8AHsmDL3yhg5Ldtl0wZpNun9WHedvgoTlXJyGq+mtlfQ9cY1WxkQgY5jDH6uz9uErnQOyA1fk+8qb03waRiCSKWISpL8ZvpcXw14fJOPkeEMk2FoNeTcJ/N6vIqm09LyLIBMDRpbh4f+zddRE5c+sfcqTse7C5swsR8qylBW6lTtkc3L+HZobcJdYA/IDgnyLEx1pzC9dmVIB2p6wW46Y2eS6/ojyUiLTO5HQ6SpzjxZSJ0=" for domain xxxxxx.xxx.xxx.com, path /, expire 0
< Set-Cookie: LtpaToken2=kfaNDGliRobAc8VL15GCnLFlr2PbXOQpkQWnO3UEBlieCQ9ShCa5db753IrowtC2JDkAaAsmSiQi0W4SdYAx1JAfA5DZgk2/Qa76xyIq8TibMe4ik0Is3xf0aVOPC62JNKqxGf56D8b3UPFiYFLmEVAgOlkMCE96JuWiG+6F4BVbih2qxFdAtipZ0pFMU5x4nFyOnKEiL6fcSiEh7Tj0JqMQoYzi2re12Ci3Y0HDUThnx+C69pczG4QrqVsbqZOfvZGgEczaFfNlV22Q7W8sH9upNifxodt74zmHbKPE7crowywTIqmtTQLaEUBzb8Ypa2H5+5u/CGjaf0KlLwzTnq+DGxSQD4PH6DkKJ5xI51rlDseFTpklx1zkAx6pEv3GlLofJnHzpFydW98F3sVZeYPWCWSPZffZbKf2NsfJDZfpf7+rRrOJxB8KeEMC3kjid9Qs+zn7E4p7eyBjzIR2J51Zx8qnBFGbv+NcdOZodTpzfQ7kg21sxzviE/qt8AHsmDL3yhg5Ldtl0wZpNun9WHedvgoTlXJyGq+mtlfQ9cY1WxkQgY5jDH6uz9uErnQOyA1fk+8qb03waRiCSKWISpL8ZvpcXw14fJOPkeEMk2FoNeTcJ/N6vIqm09LyLIBMDRpbh4f+zddRE5c+sfcqTse7C5swsR8qylBW6lTtkc3L+HZobcJdYA/IDgnyLEx1pzC9dmVIB2p6wW46Y2eS6/ojyUiLTO5HQ6SpzjxZSJ0=; Path=/
* Added cookie LtpaToken="oo/p0vusSjdPqhIWo0rNt2GNY75yMXFsia8+XFl2h6+mkxEJfyBR5Q2ktbeAM+1orWyD+v2HlkzV7Q7T6ZQdCDVkN5A3bmSRBhxfhOuS76jiIXOWf1rpbeqFWHmX1ZSXvrQLyEwpkz/+jeJ1ZvIo+1S6cj9M+MWPWibiXARjY2/o5EnZRs7G66xQ1reJLSkmv2jeQFU+NP4PuRpqfIG/m/cfn/CjGcN0VwRJmOFxCAzmqA9vQRKNZ1yN8zybVr2bOuki0QXxmEk/k1hm8SSSuIBBM6SFNsNd" for domain xxxxxx.xxx.xxx.com, path /, expire 0
< Set-Cookie: LtpaToken=oo/p0vusSjdPqhIWo0rNt2GNY75yMXFsia8+XFl2h6+mkxEJfyBR5Q2ktbeAM+1orWyD+v2HlkzV7Q7T6ZQdCDVkN5A3bmSRBhxfhOuS76jiIXOWf1rpbeqFWHmX1ZSXvrQLyEwpkz/+jeJ1ZvIo+1S6cj9M+MWPWibiXARjY2/o5EnZRs7G66xQ1reJLSkmv2jeQFU+NP4PuRpqfIG/m/cfn/CjGcN0VwRJmOFxCAzmqA9vQRKNZ1yN8zybVr2bOuki0QXxmEk/k1hm8SSSuIBBM6SFNsNd; Path=/
< Content-Length: 0
< Content-Type: text/plain
< Content-Language: en
<
* Connection #0 to host xxxxxx.xxx.xxx.com left intact
* Issue another request to this URL: 'https://xxxxx.xxx.xxx.com/sdm/'
* Violate RFC 2616/10.3.3 and switch from POST to GET
* Re-using existing connection! (#0) with host xxxxxx.xxx.xxx.com
* Connected to xxxxxx.xxx.xxx.com (200.200.200.200) port 443 (#0)
> GET /sdm/ HTTP/1.1
User-Agent: Mozilla/5.0 (compatible; MSIE 6.01; Windows NT 5.0)
Host: xxxxxx.xxx.xxx.com
(...)

------------------------------------------------
The code that connects to the SSL host is the following:

  $curl->setopt(CURLOPT_HEADER,1);
  $curl->setopt(CURLOPT_POST,1);
  $curl->setopt(CURLOPT_FOLLOWLOCATION,1);
  $curl->setopt(CURLOPT_VERBOSE,1);
  $curl->setopt(CURLOPT_SSL_VERIFYHOST,0);
  $curl->setopt(CURLOPT_SSL_VERIFYPEER,0);
  $curl->setopt(CURLOPT_COOKIEJAR,$cookiesfile);
  $curl->setopt(CURLOPT_COOKIEFILE,$cookiesfile);
  $curl->setopt(CURLOPT_USERAGENT,$useragent);
  $curl->setopt(CURLOPT_URL, $myurl);
  $curl->setopt(CURLOPT_POSTFIELDS, $mypost);
  my $response_body='';

  # NOTE - do not use a typeglob here. A reference to a typeglob is okay though.
  open (my $fileb, ">", \$response_body);
  $curl->setopt(CURLOPT_WRITEDATA,$fileb);

  if ($options{'verbose'}) { printf "**** ONLY_POST: url=%s, post=%s\n", $myurl, $mypost }
  # Starts the actual request
  my $retcode = $curl->perform;
  close ($fileb);

ProblemType: Bug
Architecture: i386
Date: Wed Dec 30 03:52:52 2009
DistroRelease: Ubuntu 9.10
Package: libwww-curl-perl 4.05-1
ProcEnviron:
 SHELL=/bin/zsh
 PATH=(custom, user)
 LANG=pt_BR.UTF-8
 LANGUAGE=pt_BR.UTF-8
ProcVersionSignature: Ubuntu 2.6.31-16.53-generic
SourcePackage: libwww-curl-perl
Uname: Linux 2.6.31-16-generic i686
XsessionErrors:
 (gnome-settings-daemon:2681): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (gnome-settings-daemon:2681): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (nautilus:2789): Eel-CRITICAL **: eel_preferences_get_boolean: assertion `preferences_is_initialized ()' failed
 (polkit-gnome-authentication-agent-1:2850): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed
 (gnome-panel:2788): Gdk-WARNING **: /build/buildd/gtk+2.0-2.18.3/gdk/x11/gdkdrawable-x11.c:952 drawable is not a pixmap or window

[Ansgar Burchardt (aburch)]

Hi,

> libwww-curl-perl 4.05-1 was compiled with openssl libs, libwww-curl-perl
> 4.07-1 is compiled without it. That breaks functionality: a program I
> developed using perl no longer works with an HTTP certificate. When I
> revert back to 4.05-1 it starts working again.

> * About to connect() to xxxxxxx.xxx.xxx.com port 443 (#0)
> * Trying 200.200.200.200... * connected
> * Connected to xxxxxxx.xxx.xxx.com (200.200.200.200) port 443 (#0)
> * found 145 certificates in /etc/ssl/certs/ca-certificates.crt
> * gnutls_handshake() failed: Decryption has failed.
> * Closing connection #0

I cannot reproduce this problem with libwww-curl-perl/4.09-1 on Debian
when running the attached test script.

Does the test script work for you? If not, is the problem still present
in Ubuntu Lucid?

Regards,
Ansgar

[Patola (patola)]

Hi, Ansgar, thanks for the response.

For most sites, the aforementioned code works. But it seems that there are some that gnutls cannot decrypt, only libssl.

I know nothing about Certificates and HTTPS encryption, but I find it might be very dependent of the site in question. The site is a corporate intranet site, so you wouldn't be able to test it anyway, but I though that, if you know something about certificates and encryption, the information of the verbose mode of libcurl can tell you something about the encryption type and certificate needed for this to work. I do not know any public sites which use this same type of certificate and encryption, and in fact I don't even know how to look for it.

I do not have the libwww-curl-perl/4.09-1, I'll try to download and test it, but I strongly suspect this is indeed caused by the lack of libssl in libwww-curl-perl. If it's absent as it was in 4.07, it won't work.

[Patola (patola)]

Just installed the libwww-curl-perl_4.09-1+b1_i386.deb package from Debian and the authentication didn't work, with the exact same error message of 4.07. Reverted back to 4.05 and it worked again, with the exact same debug information also. Here, again, is the authentication that worked:

* About to connect() to xxxxxx.xxx.xxx.com port 443 (#0)
* Trying 200.200.200.200... * connected
* Connected to xxxxxx.xxx.xxx.com (200.200.200.200) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
  CApath: /etc/ssl/certs
* SSL connection using AES256-SHA
* Server certificate:
* subject: C=US, ST=New York, L=Poughkeepsie, O=IBM, CN=xxxxxx.xxx.xxx.com
* start date: 2009-10-04 17:17:20 GMT
* expire date: 2012-10-05 22:53:18 GMT
* issuer: C=US, O=Equifax, OU=Equifax Secure Certificate Authority
* SSL certificate verify ok.
> POST /sdm/j_security_check HTTP/1.1
User-Agent: Mozilla/5.0 (compatible; MSIE 6.01; Windows NT 5.0)
Host: xxxxxx.xxx.xxx.com
Accept: */*
Content-Length: 107
Content-Type: application/x-www-form-urlencoded

Using ldd on 4.09's Curl.so, the libraries are identical to 4.07, which confirmed my suspicion that 4.09 is also compiled without libssl. So, I still think that for this authentication to work we need libssl back onto libwww-curl-perl.

[gregor herrmann (gregoa)]

On Wed, 30 Dec 2009 09:07:24 -0000, Patola wrote:

> For most sites, the aforementioned code works. But it seems that there
> are some that gnutls cannot decrypt, only libssl.

That would be a bug in gnutls then, IMO.

JFTR: The change in libwww-curl-perl happened in Debian in 4.06
because of http://bugs.debian.org/520595

Cheers,
gregor
--
 .''`. http://info.comodo.priv.at/ -- GPG Key IDs: 0x00F3CFE4, 0x8649AA06
 : :' : Debian GNU/Linux user, admin, & developer - http://www.debian.org/
 `. `' Member of VIBE!AT, SPI Inc., fellow of FSFE | http://got.to/quote/
   `- NP: Diana Krall: From This Moment On

[Patola (patola)]

It might not necesssarily be a bug. Might be just a lacking feature. In that case, the easiest solution would be to revert back to using openssl, instead of waiting for the feature to be implemented.

I must say that I do not understand what's the licensing issue here. openssl has a MIT/BSD-style license that is compatible to libcurl and WWW::Curl MIT/BSD-style licenses, isn't it? Why the need for a GPL-compatible license?

[gregor herrmann (gregoa)]

On Wed, 30 Dec 2009 19:28:24 -0000, Patola wrote:

> I must say that I do not understand what's the licensing issue here.
> openssl has a MIT/BSD-style license that is compatible to libcurl and
> WWW::Curl MIT/BSD-style licenses, isn't it? Why the need for a GPL-
> compatible license?

As noted in the mentioned Debian bug report it might be a problem for
apps/modules using WWW::Curl.

Cheers,
gregor
--
 .''`. http://info.comodo.priv.at/ -- GPG Key IDs: 0x00F3CFE4, 0x8649AA06
 : :' : Debian GNU/Linux user, admin, & developer - http://www.debian.org/
 `. `' Member of VIBE!AT, SPI Inc., fellow of FSFE | http://got.to/quote/
   `- NP: R.E.M.: King of Comedy