Segmentation fault on a slave slapd (sync replication with kerberos authentication)

Bug #1688575 reported by Suho Meso
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openldap
Fix Released
Undecided
Unassigned
openldap (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
High
Andreas Hasenack

Bug Description

[Impact]
Concurrent SASL authentications could trigger a segfault. This was observed by the bug reporter during replication from a master to a slave, and can be reproduced with a test program.

The fix is applied upstream, see comment #13.

[Test Case]
* Create a fresh xenial VM or container and login. Update the apt repositories:
sudo apt update

* Create a local directory and cd into it:
mkdir test && cd test

* Download the test attachments from this bug: Makefile, sasltest.c and testscript:
wget https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1688575/+attachment/5139678/+files/Makefile https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1688575/+attachment/5139679/+files/sasltest.c https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1688575/+attachment/5139680/+files/testscript

* Build with
  $ apt install libsasl2-dev libldap2-dev
  $ make

* Execute the testscript with sudo once. It shall fail at the very end with a core dump:
sudo sh ./testscript
(...)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
Aborted (core dumped)

* Export this var:
export LDAPSASL_SECPROPS=none

* Run the actual test script a few more times to confirm the crasH:
$ ./sasltest
rc = -6 (Unknown authentication method)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
rc = -6 (Unknown authentication method)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
rc = -6 (Unknown authentication method)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
Aborted (core dumped)

* Install the updated packages from proposed

* Run ./sasltest again. Make sure the LDAPSASL_SECPROPS var is still exported:
$ echo $LDAPSASL_SECPROPS
none

$ ./sasltest
$

This time the test completes without crashing.

[Regression Potential]
This is SASL authentication, and with kerberos nonetheless (in the case of the bug reporter). I suspect not many people deploy this due to its complexity. The ones that have such a setup, however, tend to know what they are doing, so if they say the problem is fixed for them, I believe it is.

About this particular change, it's committed upstream and also in debian, and @rtandy was kind enough to provide a sample test script that exhibits the problem.

[Other Info]
Since the fix is applied upstream and in Debian, there shouldn't be additional surprises here.

[Original description]

I have a slapd problem on a freshly installed 16.04 machine:

slapd[17107]: segfault at 1a ip 00007f3c12c79f55 sp 00007f3c03c2d080 error 4 in libsasl2.so.2.0.25[7f3c12c72000+19000]

I'm using the server as Slave LDAP-Server and sync replication with kerberos authentication.
The service either starts and runs successfully or it fails with segmentation fault or 100% CPU.
Maybe an useful info, I'm replicating two databases. When I deactivate syncrepl for one of them (doesn't matter which one) the problem is not occuring.

Linux xxx 4.4.0-75-generic #96-Ubuntu SMP Thu Apr 20 09:56:33 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
slapd 2.4.42+dfsg-2ubuntu3.1
libsasl2-2:amd64 2.1.26.dfsg1-14build1
libsasl2-modules:amd64 2.1.26.dfsg1-14build1
libsasl2-modules-gssapi-mit:amd64 2.1.26.dfsg1-14build1

GDB debug:

Starting program: /usr/sbin/slapd -h "ldap:/// ldaps:/// ldapi:///" -u openldap -g openldap -f /etc/ldap/slapd.conf -d 256
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
590c82ab @(#) $OpenLDAP: slapd (Ubuntu) (May 11 2016 16:12:05) $
 buildd@lgw01-10:/build/openldap-mF7Kfq/openldap-2.4.42+dfsg/debian/build/servers/slapd
590c82ab slapd starting
[New Thread 0x7f2e96b7b700 (LWP 42139)]
[New Thread 0x7f2e9637a700 (LWP 42140)]
[New Thread 0x7f2e95b79700 (LWP 42141)]
[New Thread 0x7f2e95378700 (LWP 42142)]
[New Thread 0x7f2e94b77700 (LWP 42143)]
590c82ba slap_client_connect: URI=ldap://xxx ldap_sasl_interactive_bind_s failed (-6)
590c82ba do_syncrepl: rid=132 rc -6 retrying (9 retries left)

Thread 4 "slapd" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f2e95b79700 (LWP 42141)]
0x00007f2ea53035b5 in sasl_client_add_plugin () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2

(gdb) thr apply all bt

Thread 6 (Thread 0x7f2e94b77700 (LWP 42143)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f2ea59463f3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#2 0x00007f2ea487c6ba in start_thread (arg=0x7f2e94b77700) at pthread_create.c:333
#3 0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 5 (Thread 0x7f2e95378700 (LWP 42142)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f2ea59463f3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#2 0x00007f2ea487c6ba in start_thread (arg=0x7f2e95378700) at pthread_create.c:333
#3 0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 4 (Thread 0x7f2e95b79700 (LWP 42141)):
#0 0x00007f2ea53035b5 in sasl_client_add_plugin () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
#1 0x00007f2ea530f250 in ?? () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
#2 0x00007f2ea5303d69 in sasl_client_init () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
#3 0x00007f2ea594da6c in ldap_int_sasl_init () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#4 0x00007f2ea594db2c in ldap_int_sasl_open () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#5 0x00007f2ea594e2d4 in ldap_int_sasl_bind () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#6 0x00007f2ea5951828 in ldap_sasl_interactive_bind () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#7 0x00007f2ea5951a4e in ldap_sasl_interactive_bind_s () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#8 0x0000561fbc556db4 in slap_client_connect (ldp=0x561fbe1e9f68, sb=0x561fbe1e9d40) at ../../../../servers/slapd/config.c:2063
#9 0x0000561fbc5c699d in do_syncrep1 (si=0x561fbe1e9d10, op=0x7f2e95b787b0) at ../../../../servers/slapd/syncrepl.c:618
#10 do_syncrepl (ctx=<optimized out>, arg=0x561fbe1e5620) at ../../../../servers/slapd/syncrepl.c:1548
#11 0x00007f2ea59463a2 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#12 0x00007f2ea487c6ba in start_thread (arg=0x7f2e95b79700) at pthread_create.c:333
#13 0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 3 (Thread 0x7f2e9637a700 (LWP 42140)):
---Type <return> to continue, or q <return> to quit---
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f2ea59463f3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
#2 0x00007f2ea487c6ba in start_thread (arg=0x7f2e9637a700) at pthread_create.c:333
#3 0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 2 (Thread 0x7f2e96b7b700 (LWP 42139)):
#0 0x00007f2ea45b2e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
#1 0x0000561fbc55a8f0 in slapd_daemon_task (ptr=<optimized out>) at ../../../../servers/slapd/daemon.c:2539
#2 0x00007f2ea487c6ba in start_thread (arg=0x7f2e96b7b700) at pthread_create.c:333
#3 0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 1 (Thread 0x7f2ea5d96740 (LWP 42138)):
#0 0x00007f2ea487d98d in pthread_join (threadid=139838073845504, thread_return=0x0) at pthread_join.c:90
#1 0x0000561fbc55cc81 in slapd_daemon () at ../../../../servers/slapd/daemon.c:2932
#2 0x0000561fbc543bea in main (argc=11, argv=<optimized out>) at ../../../../servers/slapd/main.c:1017
(gdb)

Related branches

Revision history for this message
Ryan Tandy (rtandy) wrote : Re: [Bug 1688575] [NEW] Segmentation fault on a slave slapd (sync replication with kerberos authentication)

Thanks for opening the new bug.

I have a patch under review right now for a suspiciously similar issue.

Are you able to build and test a patched package, and see if it fixes
your problem? If you need help modifying the package, I can upload it to
a PPA for you later on.

Changed in openldap (Ubuntu):
assignee: nobody → Ryan Tandy (rtandy)
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "0001-ITS-8648-add-back-mutex-for-sasl_client_init.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Ryan Tandy (rtandy)
Changed in openldap (Ubuntu):
status: New → Fix Committed
status: Fix Committed → New
Changed in openldap:
status: New → Fix Committed
Revision history for this message
Suho Meso (kunalija) wrote :

Hi Ryan,

thanks, I'll try to apply the patch by myself.
Is libldap the only package to be patched?

Revision history for this message
Suho Meso (kunalija) wrote :

I meant openldap...

Revision history for this message
Ryan Tandy (rtandy) wrote :

Yes, just openldap.

I uploaded the patched package to a PPA for you to try:

https://launchpad.net/~rtandy/+archive/ubuntu/bug1688575

Hope that helps.

Ryan Tandy (rtandy)
Changed in openldap (Ubuntu):
status: New → In Progress
Revision history for this message
Suho Meso (kunalija) wrote :

Looks good.
I've tested deleting and replicating the databases few times, 20-30 successive service restarts, no issues...

Revision history for this message
Suho Meso (kunalija) wrote :

Hi Ryan, what would be the next step? Will you integrate it in the official repo?

Revision history for this message
Ryan Tandy (rtandy) wrote : Re: [Bug 1688575] Re: Segmentation fault on a slave slapd (sync replication with kerberos authentication)

Hi,

Sorry for the silence, I'm in a busy spell and not able to look at
Ubuntu stuff right now. I do intend to follow up and propose the patch
for a stable update when I can; anyone else is welcome to beat me to it
in the meantime.

Revision history for this message
Suho Meso (kunalija) wrote :

Thanks Ryan, I will wait...

tags: added: server-next
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I can take a look at this.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Revision history for this message
Ryan Tandy (rtandy) wrote :

Hi Andreas,

On Mon, Jul 24, 2017 at 05:33:41PM -0000, Andreas Hasenack wrote:
>I can take a look at this.

Thanks. FYI the fix is released upstream in 2.4.45 and I'll be uploading
that to Debian soon.

http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commit;h=d59310f86295d5ca0e2947efc78a08448610a074
http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commit;h=71a7040393ac02c807e6f2b78967725a3ebd378f

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Thanks. I'm coming up with a test case simple enough to use for the SRU template

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Hm, I'm not getting a segfault.
I have two databases on the server: dc=example,dc=com and dc=example,dc=org. Both have syncprov, and my slave is syncrepling from both using gssapi.

I created a replicator principal, added an ACL to allow it to read everything in both trees.

I didn't use k5start in the slave, since this is just a test. I kinit'ed the replicator user, chowned the credentials cache file to openldap and set KRB5CCNAME in /etc/default/slapd.

Upon starting the slave, I get two connections from it logged on the master and their respective searches for each tree (see http://pastebin.ubuntu.com/25171586/ for full log and better formatting):
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2229 fd=13 ACCEPT from IP=10.0.100.149:60168 (IP=0.0.0.0:389)
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2230 fd=20 ACCEPT from IP=10.0.100.149:60170 (IP=0.0.0.0:389)
(...)
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2230 op=2 BIND authcid="Replicator" authzid="Replicator"
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2230 op=2 BIND dn="uid=replicator,cn=gssapi,cn=auth" mech=GSSAPI sasl_ssf=56 ssf=56
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2230 op=2 RESULT tag=97 err=0 text=
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2230 op=3 SRCH base="dc=example,dc=com" scope=2 deref=0 filter="(objectClass=*)"
(...)
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2229 op=2 BIND authcid="Replicator" authzid="Replicator"
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2229 op=2 BIND dn="uid=replicator,cn=gssapi,cn=auth" mech=GSSAPI sasl_ssf=56 ssf=56
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2229 op=2 RESULT tag=97 err=0 text=
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2229 op=3 SRCH base="dc=example,dc=org" scope=2 deref=0 filter="(objectClass=*)"
Jul 25 18:48:23 xenial-slapd-segfault-1688575 slapd[4697]: conn=2229 op=3 SRCH attr=* +

I tried multiple restarts on the slave, also between master updates with a script that creates 100 users in each tree, but no segfault.

Could you share your syncrepl and syncprov settings perhaps for both databases?

On the master I have just for each db:
olcOverlay: {0}syncprov
olcSpCheckpoint: 100 10
olcSpSessionlog: 100

And on the slave I have:
olcSyncrepl: {0}rid=0 provider=ldap://xenial-slapd-segfault-1688575.lxd bind
 method=sasl saslmech=GSSAPI searchbase="dc=example,dc=com" schemachecking=
 off type=refreshAndPersist retry="60 +"

and

olcSyncrepl: {0}rid=1 provider=ldap://xenial-slapd-segfault-1688575.lxd bind
 method=sasl saslmech=GSSAPI searchbase="dc=example,dc=org" schemachecking=
 off type=refreshAndPersist retry="60 +"

I understand it might not be an immediate segfault, or 100% cpu usage, but given the bug description I thought it was more or less constant. But maybe I'm missing something.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Just to be clear, I'm not doubting the bug, I'm just trying to come up with a test scenario that will satisfy the SRU requirements so we can ship the fix to xenial :)

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Oh, and I used libsasl2-modules-gssapi-mit and MIT kerberos KDC:
krb5-kdc 1.13.2+dfsg-5ubuntu2
libsasl2-modules-gssapi-mit:amd64 2.1.26.dfsg1-14build1

Revision history for this message
Suho Meso (kunalija) wrote :

Hi Andreas,

here are my syncprov and syncrepl configurations:

dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpCheckpoint: 5 5

dn: olcOverlay={0}syncprov,olcDatabase={2}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpCheckpoint: 5 5

olcSyncrepl: rid=201 provider=ldap://master.example.com bindmethod=sasl
 timeout=0 network-timeout=0 saslmech=GSSAPI realm=EXAMPLE.COM keepalive=0
 :0:0 starttls=yes tls_cert="/etc/ssl/certs/slave.pem" tls_key=
 "/etc/ssl/private/slave.key" tls_cacert="/etc/ssl/certs/ca-cer
 tificates.crt" tls_reqcert=demand tls_cipher_suite=NORMAL:-VERS-SSL3.0:-VER
 S-TLS-ALL:+VERS-TLS1.2:-CIPHER-ALL:-SHA1:-MD5:-RSA:+AES-256-CBC:+CAMELLIA-2
 56-CBC:+AES-128-CBC:+RSA filter="(objectclass=*)" searchbase="dc=example,dc=com" s
 cope=sub schemachecking=off type=refreshAndPersist retry="5 10 15 +"

olcSyncrepl: rid=202 provider=ldap://master.example.com bindmethod=sasl
 timeout=0 network-timeout=0 saslmech=GSSAPI realm=EXAMPLE.COM keepalive=0
 :0:0 starttls=yes tls_cert="/etc/ssl/certs/slave.pem" tls_key=
 "/etc/ssl/private/slave.key" tls_cacert="/etc/ssl/certs/ca-cer
 tificates.crt" tls_reqcert=demand tls_cipher_suite=NORMAL:-VERS-SSL3.0:-VER
 S-TLS-ALL:+VERS-TLS1.2:-CIPHER-ALL:-SHA1:-MD5:-RSA:+AES-256-CBC:+CAMELLIA-2
 56-CBC:+AES-128-CBC:+RSA filter="(objectclass=*)" searchbase="ou=db2"
 scope=sub schemachecking=off type=refreshAndPersist retry="5 10 15 +"

krb5-kdc 1.13.2+dfsg-5ubuntu2
libsasl2-modules-gssapi-mit:amd64 2.1.26.dfsg1-14build1

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (3.9 KiB)

This bug was fixed in the package openldap - 2.4.45+dfsg-1ubuntu1

---------------
openldap (2.4.45+dfsg-1ubuntu1) artful; urgency=low

  * Merge from Debian unstable. Remaining changes:
    - Enable AppArmor support:
      - d/apparmor-profile: add AppArmor profile
      - d/rules: use dh_apparmor
      - d/control: Build-Depends on dh-apparmor
      - d/slapd.README.Debian: add note about AppArmor
    - Enable GSSAPI support:
      - d/patches/gssapi.diff, thanks to Jerry Carter (Likewise):
        - Add --with-gssapi support
        - Make guess_service_principal() more robust when determining
          principal
      - d/configure.options: Configure with --with-gssapi
      - d/control: Added heimdal-dev as a build depend
      - d/rules:
        - Explicitly add -I/usr/include/heimdal to CFLAGS.
        - Explicitly add -I/usr/lib/<multiarch>/heimdal to LDFLAGS.
    - Enable ufw support:
      - d/control: suggest ufw.
      - d/rules: install ufw profile.
      - d/slapd.ufw.profile: add ufw profile.
    - Enable nss overlay:
      - d/{patches/nssov-build,rules}: Apply, build and package the
        nss overlay.
    - d/{rules,slapd.py}: Add apport hook.
    - d/slapd.init.ldif: don't set olcRootDN since it's not defined in
      either the default DIT nor via an Authn mapping.
    - d/slapd.scripts-common:
      - add slapcat_opts to local variables.
      - Fix backup directory naming for multiple reconfiguration.
    - d/{slapd.default,slapd.README.Debian}: use the new configuration style.
    - d/rules: Enable -DLDAP_CONNECTIONLESS to build CLDAP (UDP) support
      in the openldap library, as required by Likewise-Open
    - Show distribution in version:
      - d/control: added lsb-release
      - d/patches/fix-ldap-distribution.patch: show distribution in version
    - d/libldap-2.4-2.symbols: Add symbols not present in Debian.
      - CLDAP (UDP) was added in 2.4.17-1ubuntu2
      - GSSAPI support was enabled in 2.4.18-0ubuntu2

openldap (2.4.45+dfsg-1) unstable; urgency=medium

  * New upstream release.
    - fixed a use-after-free in GnuTLS options handling
      (ITS#8385) (Closes: #820244) (LP: #1557248)
    - fixed unsafe concurrent SASL calls causing memory corruption
      (ITS#8648) (Closes: #860947) (LP: #1688575)
    - fixed syncrepl infinite looping with multi-master delta-syncrepl
      (ITS#8432) (Closes: #868753)
  * Rebase patches to apply cleanly:
    - do-not-second-guess-sonames
    - no-AM_INIT_AUTOMAKE
  * Drop patches applied upstream:
    - ITS-8554-kFreeBSD-is-like-BSD.patch
    - ITS-8644-wait-for-slapd-to-start-in-test064.patch
    - ITS-8655-paged-results-double-free.patch
  * Upgrade to debhelper compat level 10.
    - Depend on debhelper 10.
    - Stop enabling parallel and autoreconf explicitly. They are now enabled
      by default.
    - Drop dh-autoreconf from build-depends since debhelper requires it.
  * Add -Wno-format-extra-args to CFLAGS to reduce the noise in the build
    logs, as this warning is emitted on each use of the Debug() macro.
  * Drop libldap-2.4-4-dbg and slapd-dbg binary packages in favour of
    automatic dbgsym packages.
  * Update Standards-Version to 4.0.0; no changes re...

Read more...

Changed in openldap (Ubuntu):
status: In Progress → Fix Released
Ryan Tandy (rtandy)
Changed in openldap:
status: Fix Committed → Fix Released
Changed in openldap (Ubuntu):
assignee: Ryan Tandy (rtandy) → nobody
Revision history for this message
Ryan Tandy (rtandy) wrote :

This slipped off my radar after the fix was uploaded to arful, but we should fix it in xenial as well.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Last I tried, I couldn't reproduce it. Can we make the case for an SRU without a clear test case?

Revision history for this message
Ryan Tandy (rtandy) wrote :

On Mon, May 14, 2018 at 02:34:13PM -0000, Andreas Hasenack wrote:
>Last I tried, I couldn't reproduce it. Can we make the case for an SRU
>without a clear test case?

I'll try and find time this week to work up instructions.

Would a program that demonstrates the issue (test instructions: compile
and run) be satisfactory for SRU purposes?

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I believe so, thanks

Revision history for this message
Ryan Tandy (rtandy) wrote :

Please find attached a test program and Makefile plus a test script to
drive it. Basically the program exercises concurrent SASL binds.

With the current packages in xenial, the test program fails in a variety
of ways:

$ ./sasltest
rc = -6 (Unknown authentication method)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
Aborted

$ ./sasltest
Segmentation fault

$ ./sasltest
Bus error

or even simply hanging/spinning.

(If you execute ./sasltest in a shell, be sure to export
LDAPSASL_SECPROPS=none first to avoid the confidentiality requirement.)

With the proposed patch, the test program should reliably complete all
its iterations (takes a few seconds) and exit successfully. I hope this
reproduces the problem for you.

My proposed debdiff and PPA are out of date and should be rebased using
the actual upstream patches. (Similar changes already landed in Debian
stretch.) I will try to take care of that this week, but if you have
everything you need and the tuits, feel free to proceed without me.

Revision history for this message
Ryan Tandy (rtandy) wrote :

I also recommend having your local hostname and FQDN in /etc/hosts when
executing that test program, as the SASL library looks it up at least
once on every iteration.

Revision history for this message
Ryan Tandy (rtandy) wrote :

The attached debdiff is basically the same as what I already uploaded to
Debian stable in 2.4.44+dfsg-5+deb9u1. No regressions were reported
against that upload.

Tested in a xenial chroot using my test program as above and the patch
fixes the issue for me.

Test packages are building now in the same PPA.
https://launchpad.net/~rtandy/+archive/ubuntu/bug1688575

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Thanks Ryan, much appreciated. I can drive this SRU.

Changed in openldap (Ubuntu Xenial):
assignee: nobody → Andreas Hasenack (ahasenack)
status: New → Triaged
importance: Undecided → High
description: updated
description: updated
description: updated
Changed in openldap (Ubuntu Xenial):
status: Triaged → In Progress
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Suho, or anyone else affected,

Accepted openldap into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openldap/2.4.42+dfsg-2ubuntu3.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in openldap (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
Revision history for this message
Suho Meso (kunalija) wrote :

Hello Brian,

I've updated 2 servers with the new package. Let me test few days, and I'll get back to you with the results.

Thanks!

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

In the meantime I ran the test from the bug description.

Before the update:
ubuntu@xenial-slaps-segfault-1688575:~/test$ export LDAPSASL_SECPROPS=none
ubuntu@xenial-slaps-segfault-1688575:~/test$ ./sasltest
rc = -6 (Unknown authentication method)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
rc = -6 (Unknown authentication method)
rc = -6 (Unknown authentication method)
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
Aborted (core dumped)

After the update:
ubuntu@xenial-slaps-segfault-1688575:~/test$ apt-cache policy libldap-2.4-2
libldap-2.4-2:
  Installed: 2.4.42+dfsg-2ubuntu3.3
  Candidate: 2.4.42+dfsg-2ubuntu3.3
  Version table:
 *** 2.4.42+dfsg-2ubuntu3.3 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2.4.42+dfsg-2ubuntu3.2 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://br.archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     2.4.42+dfsg-2ubuntu3 500
        500 http://br.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

ubuntu@xenial-slaps-segfault-1688575:~/test$ echo $LDAPSASL_SECPROPS
none
ubuntu@xenial-slaps-segfault-1688575:~/test$ ./sasltest
ubuntu@xenial-slaps-segfault-1688575:~/test$ echo $?
0

@kunalija, once your test is done, and if the new packages worked, please replace the "verification-needed-xenial" tag with "verification-done-xenial", following instructions from comment #28. Thanks!

Revision history for this message
Suho Meso (kunalija) wrote :

Hi,

this package fixes our slapd replication bug.

we tested following version:

# dpkg -l | grep 'slapd\|libldap'
ii libldap-2.4-2:amd64 2.4.42+dfsg-2ubuntu3.3 amd64 OpenLDAP libraries
ii slapd 2.4.42+dfsg-2ubuntu3.3 amd64 OpenLDAP server (slapd)

Thanks!

Suho Meso (kunalija)
tags: added: verification-done-xenial
removed: verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openldap - 2.4.42+dfsg-2ubuntu3.3

---------------
openldap (2.4.42+dfsg-2ubuntu3.3) xenial; urgency=medium

  [ Ryan Tandy ]
  * d/p/ITS-8648-check-result-of-ldap_int_initialize-in-ldap.patch,
    d/p/ITS-8648-init-SASL-library-in-global-init.patch: Import upstream
    patches to fix memory corruption caused by calling sasl_client_init()
    multiple times and possibly concurrently. (ITS#8648) (LP: #1688575)

 -- Andreas Hasenack <email address hidden> Tue, 22 May 2018 10:54:12 -0300

Changed in openldap (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Robie Basak (racb) wrote : Update Released

The verification of the Stable Release Update for openldap has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.