NTLM authentication isnt tried in libcurl3

[Bob Clough]

Binary package hint: libcurl3

On a fully updated version of Ubuntu 10.10 amd64, under both curl and php5-curl, we were unable to use NTLM authentication to talk to our exchange server. Regressing libcurl3 from 7.21.0 to 7.19.5 from karmic fixed the problem for both applications.

Curl verbose log (broken, 7.21.0):

# curl --insecure --ntlm -v -u 2008Dev.internal\\test1:test1 https://10.0.0.17/EWS/Exchange.asmx
* About to connect() to 10.0.0.17 port 443 (#0)
* Trying 10.0.0.17... connected
* Connected to 10.0.0.17 (10.0.0.17) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES128-SHA
* Server certificate:
* subject: CN=2008Exc07
* start date: 2010-09-19 23:27:30 GMT
* expire date: 2011-09-19 23:27:30 GMT
* common name: 2008Exc07 (does not match '10.0.0.17')
* issuer: CN=2008Exc07
* SSL certificate verify result: unable to get local issuer
certificate (20), continuing anyway.
* Server auth using NTLM with user '2008Dev.internal\test1'
> GET /EWS/Exchange.asmx HTTP/1.1
> Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=
> User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
> Host: 10.0.0.17
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Server: Microsoft-IIS/7.0
< WWW-Authenticate: NTLM TlRMTVNTUAACAAAABwAHADgAAAAGgokC/XqDA0P38r0AAAAAAAAAALgAuAA/AAAABgByFwAAAA8yMDA4REVWAgAOADIAMAAwADgARABFAFYAAQASADIAMAAwADgARQBYAEMAMAA3AAQAIAAyADAAMAA4AGQAZQB2AC4AaQBuAHQAZQByAG4AYQBsAAMANAAyADAAMAA4AEUAeABjADAANwAuADIAMAAwADgAZABlAHYALgBpAG4AdABlAHIAbgBhAGwABQAgADIAMAAwADgAZABlAHYALgBpAG4AdABlAHIAbgBhAGwABwAIAAm9m+t+hcsBAAAAAA==
* gss_init_sec_context() failed: : Cannot determine realm for numeric host address
< WWW-Authenticate: Negotiate
< X-Powered-By: ASP.NET
< Date: Tue, 16 Nov 2010 11:10:46 GMT
< Content-Length: 0
<
* Connection #0 to host 10.0.0.17 left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

Curl verbose log (7.19.5, working version)

# curl --insecure --ntlm -v -u 2008Dev.internal\\test1:test1 https://10.0.0.17/EWS/Exchange.asmx
* About to connect() to 10.0.0.17 port 443 (#0)
* Trying 10.0.0.17... connected
* Connected to 10.0.0.17 (10.0.0.17) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES128-SHA
* Server certificate:
* subject: CN=2008Exc07
* start date: 2010-09-19 23:27:30 GMT
* expire date: 2011-09-19 23:27:30 GMT
* common name: 2008Exc07 (does not match '10.0.0.17')
* issuer: CN=2008Exc07
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Server auth using NTLM with user '2008Dev.internal\test1'
> GET /EWS/Exchange.asmx HTTP/1.1
> Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=
> User-Agent: curl/7.19.5 (x86_64-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
> Host: 10.0.0.17
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Server: Microsoft-IIS/7.0
< WWW-Authenticate: NTLM
TlRMTVNTUAACAAAABwAHADgAAAAGgokC5tMN7bhBbsAAAAAAAAAAALgAuAA/AAAABgByFwAAAA8yMDA4REVWAgAOADIAMAAwADgARABFAFYAAQASADIAMAAwADgARQBYAEMAMAA3AAQAIAAyADAAMAA4AGQAZQB2AC4AaQBuAHQAZQByAG4AYQBsAAMANAAyADAAMAA4AEUAeABjADAANwAuADIAMAAwADgAZABlAHYALgBpAG4AdABlAHIAbgBhAGwABQAgADIAMAAwADgAZABlAHYALgBpAG4AdABlAHIAbgBhAGwABwAIACUNUIqFhcsBAAAAAA==
* gss_init_sec_context() failed: : Credentials cache file '/tmp/krb5cc_1000' not found
< WWW-Authenticate: Negotiate
< X-Powered-By: ASP.NET
< Date: Tue, 16 Nov 2010 11:58:10 GMT
< Content-Length: 0
<
* Connection #0 to host 10.0.0.17 left intact
* Issue another request to this URL: 'https://10.0.0.17/EWS/Exchange.asmx'
* Re-using existing connection! (#0) with host 10.0.0.17
* Connected to 10.0.0.17 (10.0.0.17) port 443 (#0)
* Server auth using NTLM with user '2008Dev.internal\test1'
> GET /EWS/Exchange.asmx HTTP/1.1
> Authorization: NTLM
TlRMTVNTUAADAAAAGAAYAEAAAAAYABgAWAAAABAAEABwAAAABQAFAIAAAAANAA0AhQAAAAAAAAAAAAAABoKJAqEGJrSulZ+8AAAAAAAAAAAAAAAAAAAAACxk4WklyuRftTIFrxWQy3VJi7znhmcDezIwMDhEZXYuaW50ZXJuYWx0ZXN0MXBtY25hbGx5LWlNYWM=
> User-Agent: curl/7.19.5 (x86_64-pc-linux-gnu) libcurl/7.19.5
OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
> Host: 10.0.0.17
> Accept: */*
>
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html
< Location: /EWS/Services.wsdl
< Server: Microsoft-IIS/7.0
< X-AspNet-Version: 2.0.50727
< X-Powered-By: ASP.NET
< Date: Tue, 16 Nov 2010 11:58:10 GMT
< Connection: close
<
<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fEWS%2fServices.wsdl">here</a>.</h2>
</body></html>
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

[Jeff Runyan (jeff-alliedstrategy)]

Our business encountered the same issue during a server transition. Same errors encountered as shown in the verbose curl output above.

On a fully updated version of Ubuntu 10.10 amd64, using curl, we were unable to use NTLM authentication to talk to our Exchange 2010 Server.

As suggested by Bob above, regressing libcurl3 from 7.21.0 to 7.19.5 from karmic fixed the problem!

THANK YOU BOB! Now hopefully we can eventually re-upgrade curl!

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Anders Sandblad (arune)]

As Karmic is not available anymore I do not know how to regress versions, Ive tried installing libcurl3 from Hardy but got dependency problems with php5-curl (depending on phpapi-20060613+lfs).

Any ideas?

[Vadim S. (vadim-cuuma)]

Why would you want to install it on hardy? This bug does not apply to hardy.

Bug is present in lucid or higher. At least for me.

I install .deb package from karmic to make NTLM work again in lucid.

- Vade
> As Karmic is not available anymore I do not know how to regress
> versions, Ive tried installing libcurl3 from Hardy but got dependency
> problems with php5-curl (depending on phpapi-20060613+lfs).
>
> Any ideas?
>

[Anders Sandblad (arune)]

Read my comment again. Im trying to install the version FROM hardy since karmic is no longer available. Im also running lucid.

[Anders Sandblad (arune)]

I noticed that when trying curl with a sharepoint site it works with older versions of curl!

curl --version
curl 7.21.0 (i686-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
Protocols: dict file ftp ftps http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

Here is the command I run when connecting to an exchange server:
curl --ntlm --verbose --show-error --user xxx https://server.com/EWS/Services.wsdl > /tmp/test

Here is the command I run when connecting to a sharepoint server:
curl --ntlm --verbose --show-error --user xxx https://server.com/Lists/Kontakter/AllItems.aspx > /tmp/test

The big differences I notice in the verbose outputs are:
For sharepoint:
Server: Microsoft-IIS/6.0
Content-Length: 1539

For exchange:
Server: Microsoft-IIS/7.0
Content-Length: 0
The string "gss_init_sec_context() failed: : Credentials cache file '/tmp/krb5cc_1000' not found" is shown
The string "WWW-Authenticate: Negotiate" is shown

Also the returned WWW-Authenticate base64 encoded data differs, but maybe the server name in someway is encoded.

[Vadim S. (vadim-cuuma)]

Sorry for my stupidity man. :)

Download it from here (this is what i use in lucid):

amd64:
http://launchpadlibrarian.net/30289951/libcurl3-gnutls_7.19.5-1ubuntu2_amd64.deb

i386:
http://launchpadlibrarian.net/30287711/libcurl3-gnutls_7.19.5-1ubuntu2_i386.deb

- Vadim
> Read my comment again. Im trying to install the version FROM hardy since
> karmic is no longer available. Im also running lucid.
>

[Anders Sandblad (arune)]

Thanks, that helped me, I also installed curl and libcurl3 version 7.19.5. Pinned packages in /etc/apt/preferences:

Package: libcurl3
Pin: version 7.19.5-1ubuntu2
Pin-Priority: 1001

Package: libcurl3-gnutls
Pin: version 7.19.5-1ubuntu2
Pin-Priority: 1001

Package: curl
Pin: version 7.19.5-1ubuntu2
Pin-Priority: 1001

[Jamalulkhair (jalut78)]

Thanks for the solution. Been banging my head for 2 days try to figure out what is wrong with my PHP code which is working fine in Arch Linux but failed when tested in Ubuntu 10.04 and 11.04.

Problems resolved after downgrading libcurl3, libcurl3-gnutls & curl to version 7.19.5 (as suggested above) on those ubuntu machines.

[Martin Heide (martin-heide)]

Hi! I'm having the same issue in Ubuntu 12.04.4 (precise). (Same PHP code / curl commandline works fine in 13.04.) All my packages are up to date.

Is there any chance that there will be a fix in the official package sources, since 12.04 is an LTS release and still supported? Maybe those M$ proprietery protocols are not so popular under linux, but a fix would be nice to have! :-)

[Martin Heide (martin-heide)]

This commit from 2011 fixes it: https://github.com/bagder/curl/commit/4851dafcf164bf2de5bd33c3cf2b786422ed05b6
(basically just an if-statement around everything, the rest is indentation).

Built the curl source package of Ubuntu 12.04 with and without that fix to verify.
Would be nice if someone could make an update for 12.04 from this!

[Sorin Sbarnea (ssbarnea)]

This bug affects lots of other products and prevents us from using curl to test availability of different web services (affecting zabbix, nagios, ...).

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.