slapd crash with pwdAccountLockedTime and stacked overlays

Bug #1866303 reported by Ryan Tandy on 2020-03-06
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openldap (Debian)
Fix Released
Unknown
openldap (Ubuntu)
Undecided
Andreas Hasenack
Xenial
Undecided
Andreas Hasenack
Bionic
Undecided
Andreas Hasenack
Disco
Undecided
Unassigned
Eoan
Undecided
Andreas Hasenack

Bug Description

[Impact]
In the configuration and conditions described below, slapd can crash:

1. ppolicy overlay configured with pwdLockout: TRUE
2. smbk5pwd overlay stacked after ppolicy
3. an account locked out via pwdAccountLockedTime
4. a client binding to the locked-out account and also requesting the ppolicy control

[Test Case]

* get the files from the bug:
mkdir slapd-test-case; cd slapd-test-case
wget -ct0 https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1866303/+attachment/5334194/+files/slapd.conf https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1866303/+attachment/5334195/+files/data.ldif https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1866303/+attachment/5334196/+files/samba.schema https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1866303/+attachment/5334197/+files/script

* run the script:
sudo apt update && sudo sh ./script

* With the bug, the result is:
ldap_bind: Invalid credentials (49)
slapd dead

* If when confirming the bug you don't see "slapd dead" like above, check manually, as slapd might have been in the process of shutting down when the script checked its status: "sudo systemctl status slapd"

* With the fixed packages, you get a living slapd at the end (you can run the script again on the same system after updating the packages):

sudo sh ./script
...
slapd running
ldap_bind: Invalid credentials (49)
slapd running

[Regression Potential]
The fix is in the password policy overlay (not enabled by default), so any regressions would be around that area and could potentially impact authentication ("binding") to openldap.

[Other Info]
This was fixed in focal and "cooked" there for a long while, as suggested by the Debian maintainer. We haven't received further bug reports about this in focal+.

[Original Description]

Hello,

Please merge openldap 2.4.49+dfsg-2 from Debian unstable to fix an issue in the ppolicy overlay that can crash slapd. Please also consider SRUing the patch after it has had some testing time.

Upstream: https://openldap.org/its/?findid=9171
Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953150

The ingredients for the crash are:

1: ppolicy overlay configured with pwdLockout: TRUE
2. smbk5pwd overlay stacked after ppolicy
3. an account locked out via pwdAccountLockedTime
4. a client binding to the locked-out account and also requesting the ppolicy control

The buggy code is not as specific as the above steps, so I suspect there are probably other configurations or steps that can trigger the same crash.

I will attach my test script and data for reproducing the crash.

Expected output (last lines):

[ ok ] Starting OpenLDAP: slapd.
slapd running
ldap_bind: Invalid credentials (49)
slapd running

Actual output (last lines):

[ ok ] Starting OpenLDAP: slapd.
slapd running
ldap_bind: Invalid credentials (49)
slapd dead

Related branches

Changed in openldap (Ubuntu):
status: New → In Progress
assignee: nobody → Andreas Hasenack (ahasenack)
Andreas Hasenack (ahasenack) wrote :

Thanks a lot for this Ryan, and awesome testing script!

Robie Basak (racb) on 2020-03-09
tags: added: server-next
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openldap - 2.4.49+dfsg-2ubuntu1

---------------
openldap (2.4.49+dfsg-2ubuntu1) focal; urgency=medium

  * Merge with Debian unstable (LP: #1866303). Remaining changes:
    - Enable AppArmor support:
      - d/apparmor-profile: add AppArmor profile
      - d/rules: use dh_apparmor
      - d/control: Build-Depends on dh-apparmor
      - d/slapd.README.Debian: add note about AppArmor
    - Enable GSSAPI support:
      - d/patches/gssapi.diff, thanks to Jerry Carter (Likewise):
        - Add --with-gssapi support
        - Make guess_service_principal() more robust when determining
          principal
        [Dropped the ldap_gssapi_bind_s() hunk as that is already
      - d/configure.options: Configure with --with-gssapi
      - d/control: Added heimdal-dev as a build depend
      - d/rules:
        - Explicitly add -I/usr/include/heimdal to CFLAGS.
        - Explicitly add -I/usr/lib/<multiarch>/heimdal to LDFLAGS.
    - Enable ufw support:
      - d/control: suggest ufw.
      - d/rules: install ufw profile.
      - d/slapd.ufw.profile: add ufw profile.
    - Enable nss overlay:
      - d/rules:
        - add nssov to CONTRIB_MODULES
        - add sysconfdir to CONTRIB_MAKEVARS
      - d/slapd.install:
        - install nssov overlay
      - d/slapd.manpages:
        - install slapo-nssov(5) man page
    - d/{rules,slapd.py}: Add apport hook.
    - d/slapd.init.ldif: don't set olcRootDN since it's not defined in
      either the default DIT nor via an Authn mapping.
    - d/slapd.scripts-common:
      - add slapcat_opts to local variables.
      - Fix backup directory naming for multiple reconfiguration.
    - d/{slapd.default,slapd.README.Debian}: use the new configuration style.
    - d/rules: Enable -DLDAP_CONNECTIONLESS to build CLDAP (UDP) support
      in the openldap library, as required by Likewise-Open
    - Show distribution in version:
      - d/control: added lsb-release
      - d/patches/fix-ldap-distribution.patch: show distribution in version
    - d/libldap-2.4-2.symbols: Add symbols not present in Debian.
      - CLDAP (UDP) was added in 2.4.17-1ubuntu2
      - GSSAPI support was enabled in 2.4.18-0ubuntu2
    - d/p/contrib-makefiles: given the change in 2.4.47+dfsg-3 regarding
      Debian bug #919136, we also have to patch the nssov makefile
      accordingly and thus update this patch.

openldap (2.4.49+dfsg-2) unstable; urgency=medium

  * slapd.README.Debian: Document the initial setup performed by slapd's
    maintainer scripts in more detail. Thanks to Karl O. Pinc.
    (Closes: #952501)
  * Import upstream patch to fix slapd crashing in certain configurations when
    a client attempts a login to a locked account.
    (ITS#9171) (Closes: #953150)

 -- Andreas Hasenack <email address hidden> Fri, 06 Mar 2020 11:39:12 -0300

Changed in openldap (Ubuntu):
status: In Progress → Fix Released
Bryce Harrington (bryce) wrote :

We're no longer looking at backporting fixes for disco.

This looks suitable for SRU so the other proposed series tasks are valid, and this is already in the server-next queue.

Changed in openldap (Ubuntu Disco):
status: New → Won't Fix
Changed in openldap (Debian):
status: Unknown → Fix Released
Andreas Hasenack (ahasenack) wrote :

This fix was added to focal, and we haven't received any crash reports about it as far as I know, so I'm proceeding with the SRU for the other ubuntu releases.

Changed in openldap (Ubuntu Xenial):
status: New → In Progress
assignee: nobody → Andreas Hasenack (ahasenack)
Changed in openldap (Ubuntu Bionic):
status: New → In Progress
assignee: nobody → Andreas Hasenack (ahasenack)
Changed in openldap (Ubuntu Eoan):
status: New → In Progress
assignee: nobody → Andreas Hasenack (ahasenack)
description: updated
description: updated

Hello Ryan, or anyone else affected,

Accepted openldap into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openldap/2.4.48+dfsg-1ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in openldap (Ubuntu Eoan):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-eoan
Changed in openldap (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Brian Murray (brian-murray) wrote :

Hello Ryan, or anyone else affected,

Accepted openldap into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openldap/2.4.45+dfsg-1ubuntu1.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in openldap (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Brian Murray (brian-murray) wrote :

Hello Ryan, or anyone else affected,

Accepted openldap into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openldap/2.4.42+dfsg-2ubuntu3.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted openldap (2.4.48+dfsg-1ubuntu1.2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

asterisk/1:16.2.1~dfsg-2build2 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#openldap

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Andreas Hasenack (ahasenack) wrote :

Xenial verification

Reproducing the error:
root@xenial-openldap-saslauthd-1557157:~# ldapsearch -H ldapi:/// -LLL -b 'dc=example,dc=com' -s base -U root -Y PLAIN
SASL/PLAIN authentication started
Please enter your password:
ldap_sasl_interactive_bind_s: Other (e.g., implementation specific) error (80)
 additional info: SASL(-1): generic failure: Password verification failed

And dmesg:
[qua jul 8 11:50:42 2020] audit: type=1400 audit(1594219843.513:405): apparmor="DENIED" operation="connect" namespace="root//lxd-xenial-openldap-saslauthd-1557157_<var-snap-lxd-common-lxd>" profile="/usr/sbin/slapd" name="/run/saslauthd/mux" pid=83468 comm="slapd" requested_mask="wr" denied_mask="wr" fsuid=1000112 ouid=1000000

With the updated packages, ldapsearch works:
root@xenial-openldap-saslauthd-1557157:~# apt-cache policy slapd
slapd:
  Installed: 2.4.42+dfsg-2ubuntu3.9
  Candidate: 2.4.42+dfsg-2ubuntu3.9
  Version table:
 *** 2.4.42+dfsg-2ubuntu3.9 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
...

root@xenial-openldap-saslauthd-1557157:~# ldapsearch -H ldapi:/// -LLL -b 'dc=example,dc=com' -s base -U root -Y PLAIN
SASL/PLAIN authentication started
Please enter your password:
SASL username: root
SASL SSF: 0
dn: dc=example,dc=com
objectClass: top
objectClass: dcObject
objectClass: organization
o: example
dc: example

And no dmesg apparmor error.

Xenial verification succeeded.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Andreas Hasenack (ahasenack) wrote :

I'm sorry, the above verification was for the other bug that this upload is fixing.

tags: added: verification-needed-xenial
removed: verification-done-xenial
Andreas Hasenack (ahasenack) wrote :

Xenial verification (for real)

Reproducing the bug:
  Version table:
 *** 2.4.42+dfsg-2ubuntu3.8 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://br.archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status

$ sudo sh ./script
...
Closing DB...
slapd running
ldap_bind: Invalid credentials (49)
slapd dead

With the packages from proposed, slapd remains running:
  Version table:
 *** 2.4.42+dfsg-2ubuntu3.9 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

$ sudo sh ./script
...
Closing DB...
slapd running
ldap_bind: Invalid credentials (49)
slapd running

Xenial verification succeeded.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Andreas Hasenack (ahasenack) wrote :

Bionic verification

Reproducing the bug:
  Version table:
 *** 2.4.45+dfsg-1ubuntu1.5 500
        500 http://br.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        500 http://br.archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages
        100 /var/lib/dpkg/status

$ sudo sh ./script
...
Closing DB...
slapd running
ldap_bind: Invalid credentials (49)
slapd dead

Updating to proposed:
  Version table:
 *** 2.4.45+dfsg-1ubuntu1.6 500
        500 http://br.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

Now slapd remains running:
$ sudo sh ./script
...
Closing DB...
slapd running
ldap_bind: Invalid credentials (49)
slapd running

Bionic verification succeeded.

tags: added: verification-done-bionic
removed: verification-needed-bionic
Andreas Hasenack (ahasenack) wrote :

Eoan verification

Reproducing the problem:
  Version table:
 *** 2.4.48+dfsg-1ubuntu1.1 500
        500 http://br.archive.ubuntu.com/ubuntu eoan-updates/main amd64 Packages
        500 http://br.archive.ubuntu.com/ubuntu eoan-security/main amd64 Packages
        100 /var/lib/dpkg/status

ubuntu@eoan-openldap-crash-1866303:~/slapd-test-case$ sudo sh ./script
...
Closing DB...
slapd running
ldap_bind: Invalid credentials (49)
slapd dead

With the proposed packages:
  Version table:
 *** 2.4.48+dfsg-1ubuntu1.2 500
        500 http://br.archive.ubuntu.com/ubuntu eoan-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

slapd remains running:
ubuntu@eoan-openldap-crash-1866303:~/slapd-test-case$ sudo sh ./script
...
Closing DB...
slapd running
ldap_bind: Invalid credentials (49)
slapd running

Eoan verification succeeded.

tags: added: verification-done-eoan
removed: verification-needed-eoan
Andreas Hasenack (ahasenack) wrote :

The asterisk DEP8 armhf test was retried and is now green.

Andreas Hasenack (ahasenack) wrote :

Kopanocore armhf is the only persistent red, but this test/package is known to be flaky on armhf.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openldap - 2.4.48+dfsg-1ubuntu1.2

---------------
openldap (2.4.48+dfsg-1ubuntu1.2) eoan; urgency=medium

  [ Andreas Hasenack ]
  * d/p/ITS-9171-Insert-callback-in-the-right-place.patch: Import upstream
    patch to fix slapd crashing in certain configurations when a client
    attempts a login to a locked account. (LP: #1866303)

  [ Sergio Durigan Junior ]
  * d/apparmor-profile: Update apparmor profile to grant access to
    the saslauthd socket, so that SASL authentication works. (LP: #1557157)

 -- Andreas Hasenack <email address hidden> Wed, 01 Jul 2020 16:43:06 -0300

Changed in openldap (Ubuntu Eoan):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for openldap has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openldap - 2.4.45+dfsg-1ubuntu1.6

---------------
openldap (2.4.45+dfsg-1ubuntu1.6) bionic; urgency=medium

  [ Andreas Hasenack ]
  * d/p/ITS-9171-Insert-callback-in-the-right-place.patch: Import upstream
    patch to fix slapd crashing in certain configurations when a client
    attempts a login to a locked account. (LP: #1866303)

  [ Sergio Durigan Junior ]
  * d/apparmor-profile: Update apparmor profile to grant access to
    the saslauthd socket, so that SASL authentication works. (LP: #1557157)

 -- Andreas Hasenack <email address hidden> Wed, 01 Jul 2020 16:38:55 -0300

Changed in openldap (Ubuntu Bionic):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openldap - 2.4.42+dfsg-2ubuntu3.9

---------------
openldap (2.4.42+dfsg-2ubuntu3.9) xenial; urgency=medium

  [ Andreas Hasenack ]
  * d/p/ITS-9171-Insert-callback-in-the-right-place.patch: Import upstream
    patch to fix slapd crashing in certain configurations when a client
    attempts a login to a locked account. (LP: #1866303)

  [ Sergio Durigan Junior]
  * d/apparmor-profile: Update apparmor profile to grant access to
    the saslauthd socket, so that SASL authentication works. (LP: #1557157)

 -- Andreas Hasenack <email address hidden> Wed, 01 Jul 2020 16:33:08 -0300

Changed in openldap (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.