LDAP Authentication problem : ldap_simple_bind_s() failed

[David Goulet]

Here's the deal :
System : Ubuntu 8.04 Hardy
Linux Kernel OpenVZ 2.6.24-19.41
Apache 2.2.8-1ubuntu0.3 using the LDAP module for authentication.
LDAP server (slapd) 2.4.9-0ubuntu0.8.04.1 using TLS.

The bug :
Apache configuration for a specific location for LDAP Auth :
This example is taken from the SVN server. But this bug also happen on my Nagios server and all other servers that use mod_ldap of Apache.

<Location /svn/Config>

    DAV svn
    SVNPath /var/svn/Config

    AuthName "SVN Repository"
    AuthType Basic
    Require user svnadmin

    AuthBasicProvider file ldap
    AuthzLDAPAuthoritative off

    AuthUserFile /etc/subversion/svn-auth/users

    AuthLDAPURL ldaps://LDAP_SERVER:636/dc=MY_DC,dc=com
    AuthLDAPBindDN "SOME_DN"
    AuthLDAPBindPassword "SOMETHING"
    AuthLDAPGroupAttribute memberUid
    AuthLDAPGroupAttributeIsDN off

    Require ldap-attribute gidNumber=1004

    Require ldap-group SOME_CN

    #Satisfy any
</Location>

Here where it gets interesting. We start Apache and it works very fine! Authentication is working good and everything is fine. After a while, authentication failed to work and I'm stuck with a 500 Internal Server Error. Here is the ssl-error Apache log error message :

[Wed Dec 10 11:01:42 2008] [warn] [client 192.168.1.1] [382] auth_ldap authenticate: user foo authentication failed; URI /svn/Config/ [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP server]

But the LDAP server is up and running good. From the server (svn server), I can make successful ldap_search and I'm right now using LDAP for ssh auth so there should not be a problem with contacting the LDAP server. Here's on the LDAP server side :

Dec 10 11:01:42 ldap slapd[19479]: conn=20 fd=27 ACCEPT from IP=192.168.1.31:40521 (IP=0.0.0.0:636)
Dec 10 11:01:42 ldap slapd[19479]: conn=20 fd=27 TLS established tls_ssf=32 ssf=32
Dec 10 11:01:42 ldap slapd[19479]: conn=20 fd=27 closed (connection lost)

According to the source code, mod_ldap of Apache tries 10 times to authenticate the user and then return error. The previous message happen indeed 10 times in my log.

At this point, if I reload or restart Apache, it will fix the problem for a short while. So they might be various problems on persistent connection and/or simple bind.

Thanks to all

[Mathias Gug (mathiaz)]

Hi David,

On Wed, Dec 10, 2008 at 04:24:49PM -0000, David Goulet wrote:
> Public bug reported:
>
> Here's the deal :
> System : Ubuntu 8.04 Hardy
> Linux Kernel OpenVZ 2.6.24-19.41

Does this happen only with an OpenVZ kernel? Do you see any message in
the kernel log files?

  status incomplete

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

[goto (gotolaunchpad)]

I'm not quite sure if it's the same thing as is happening here, but we've basically just had our entire infrastructure broken because it relies on LDAP over SSL.

It seems that the newest release of libgnutls13 (2.0.4-1ubuntu2.3), which was supposed to fix a " where some valid certificate chains would be untrusted", ironically breaks our (valid) certificate chain.

The following is a log of a machine which had not yet been updated, basically enabling me to point the finger at libgnutls:

root@xibalba246:~# apt-cache policy libgnutls13
libgnutls13:
  Installed: 2.0.4-1ubuntu2.1
  Candidate: 2.0.4-1ubuntu2.3
  Version table:
     2.0.4-1ubuntu2.3 0
        500 http://ch.archive.ubuntu.com hardy-updates/main Packages
        500 http://security.ubuntu.com hardy-security/main Packages
 *** 2.0.4-1ubuntu2.1 0
        100 /var/lib/dpkg/status
     2.0.4-1ubuntu2 0
        500 http://ch.archive.ubuntu.com hardy/main Packages
root@xibalba246:~# ldapsearch -x -H ldaps://xibalba233.cs.unibas.ch/ -D cn=xxxcensored,dc=dbis,dc=cs,dc=unibas,dc=ch -w xxxcensored -b "dc=dbis,dc=cs,dc=unibas,dc=ch" -s sub '(&(objectclass=posixAccount)(uid=xxxcensored))'
# extended LDIF
#
# LDAPv3
# base <dc=dbis,dc=cs,dc=unibas,dc=ch> with scope subtree
# filter: (&(objectclass=posixAccount)(uid=xxxcensored))
# requesting: ALL
#

[...]

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

root@xibalba246:~# apt-get -y -q install libgnutls13
Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
  gnutls-bin
The following packages will be upgraded:
  libgnutls13
1 upgraded, 0 newly installed, 0 to remove and 56 not upgraded.
Need to get 0B/307kB of archives.
After this operation, 0B of additional disk space will be used.
(Reading database ... 24099 files and directories currently installed.)
Preparing to replace libgnutls13 2.0.4-1ubuntu2.1 (using .../libgnutls13_2.0.4-1ubuntu2.3_i386.deb) ...
Unpacking replacement libgnutls13 ...
Setting up libgnutls13 (2.0.4-1ubuntu2.3) ...

Processing triggers for libc6 ...
ldconfig deferred processing now taking place
root@xibalba246:~# ldapsearch -x -H ldaps://xibalba233.cs.unibas.ch/ -D cn=xxxcensored,dc=dbis,dc=cs,dc=unibas,dc=ch -w xxxcensored -b "dc=dbis,dc=cs,dc=unibas,dc=ch" -s sub '(&(objectclass=posixAccount)(uid=xxxcensored))'
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)

root@xibalba246:~#

As said, since our user management is in LDAP (pam_ldap), we're currently losing our servers one by one ;-)

Anyway, these machines are not publicly accessible, so you won't be able to immediately test it for yourself. However, there is a *web* server which is using the same certificate chain (just a different CN), available at https://dbis.cs.unibas.ch/

If you contact anybody from the team there (at the mentioned website, that is), we will gladly help you with more details, if required.

One final note: I'm definitely sure that the certificate chain as absolutely correct (and none of them is about to expire anytime soon). This setup has been working for months now in all kinds of SSL/TLS-encrypted settings.

We'd be happy to see this fixed :-D

[David Goulet (dgoulet)]

The new gnutls13 update, released yesterday (11/12/2008) fixed the problem.

[Steve Wilson (stevew)]

I had the same problem as originally posted. I am running a Subversion server in an OpenVZ container (2.6.24-21-openvz) with latest updates to libgnutls13. My Subversion server also authenticates using LDAP SSL. Initially, authentication would work correctly but then after a period of time it would start failing with errors like the following showing up in the Apache logs:
    [Fri Dec 12 17:17:14 2008] [warn] [client XXX.XXX.XXX.XXX] [17128] auth_ldap authenticate: user XXX authentication failed; URI /svn/project [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP server]

Once LDAP authentication began failing, it required a reboot of the Apache service to function normally again.

Reverting to a previous libgnutls13 (2.0.4-1ubuntu2 instead of 2.0.4-1ubuntu2.3) seemed to correct the problem. But I still saw unusual messages like:
    [Wed Dec 17 09:41:53 2008] [warn] [client XXX.XXX.XXX.XXX] [9506] auth_ldap authenticate: user XXX authentication failed; URI /svn/project [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP server]
when invalid credentials were supplied.

I noticed this post:
    http://the.unwashedmeme.com/blog/2008/08/08/mod_ldap-ldapverifyservercert-simple-bind-failed/
and decided to try disabling LDAP server verification in the Apache configuration (default is enabled):
    LDAPVerifyServerCert Off

That seems to have corrected the problem for me. I am now up to the current version of libgnutls13, LDAP authentication works, and I don't see "Can't contact LDAP server" messages in my logs any more.

So, perhaps the original problem is due to there being something wrong with the LDAP server certificates (e.g., the file is missing, the permissions are incorrect, etc.).

Hope that's of some help!

Steve

[Chuck Short (zulcss)]

This looks like is fixed according to the bug report.

[Pete Deremer (sportman1280-deactivatedaccount)]

We are experiencing this exact problem with Hardy. libgnutils13-ubuntu2.6. The weird part is that one server works and the other doesn't despite us literally copying the configurations from the working server to the non-working server, and it not improving anything.