The important backtrace in there is the one from thread 11:
#0 0x00007fb288428474 in read () from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x00007fb2890c4518 in ?? () from /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2
No symbol table info available.
#2 0x00007fb287895848 in ?? () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30
No symbol table info available.
#3 0x00007fb28788f96a in ?? () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30
No symbol table info available.
#4 0x00007fb287896d03 in ?? () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30
No symbol table info available.
#5 0x00007fb28789991c in ?? () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30
No symbol table info available.
#6 0x00007fb2878a10cb in ?? () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30
No symbol table info available.
#7 0x00007fb28789d572 in gnutls_handshake () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30
No symbol table info available.
#8 0x00007fb289304199 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
No symbol table info available.
#9 0x00007fb289301abb in ldap_pvt_tls_accept () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
No symbol table info available.
#10 0x0000556b843e6f69 in connection_read (cri=<synthetic pointer>, s=430) at ../../../../servers/slapd/connection.c:1375
debug symbols are missing there, but I have the exact same problem and get:
#0 0x00007f2a01101474 in __libc_read (fd=40, buf=0x7f29dc142ecb, nbytes=5) at ../sysdeps/unix/sysv/linux/read.c:27
#1 0x00007f2a01db0518 in sb_debug_read (sbiod=0x7f29dc10e940, buf=0x7f29dc142ecb, len=5) at ../../../../libraries/liblber/sockbuf.c:829
#2 0x00007f2a00558848 in _gnutls_stream_read (ms=0x7f29e8ffb41c, pull_func=0x7f2a01ff1da0 <tlsg_recv>, size=5, bufel=<synthetic pointer>, session=0x7f29dc008060) at buffers.c:344
#3 _gnutls_read (ms=0x7f29e8ffb41c, pull_func=0x7f2a01ff1da0 <tlsg_recv>, size=5, bufel=<synthetic pointer>, session=0x7f29dc008060) at buffers.c:424
#4 _gnutls_io_read_buffered (session=session@entry=0x7f29dc008060, total=5, recv_type=recv_type@entry=4294967295, ms=0x7f29e8ffb41c) at buffers.c:579
#5 0x00007f2a0055296a in recv_headers (ms=<optimized out>, record=0x7f29e8ffb470, htype=GNUTLS_HANDSHAKE_CLIENT_KEY_EXCHANGE, type=GNUTLS_HANDSHAKE, record_params=0x7f29dc1279f0, session=0x7f29dc008060) at record.c:1045
#6 _gnutls_recv_in_buffers (session=session@entry=0x7f29dc008060, type=type@entry=GNUTLS_HANDSHAKE, htype=htype@entry=GNUTLS_HANDSHAKE_CLIENT_KEY_EXCHANGE, ms=<optimized out>, ms@entry=0) at record.c:1173
#7 0x00007f2a00559d03 in _gnutls_handshake_io_recv_int (session=session@entry=0x7f29dc008060, htype=htype@entry=GNUTLS_HANDSHAKE_CLIENT_KEY_EXCHANGE, hsk=hsk@entry=0x7f29e8ffb580, optional=optional@entry=0) at buffers.c:1412
#8 0x00007f2a0055c91c in _gnutls_recv_handshake (session=session@entry=0x7f29dc008060, type=type@entry=GNUTLS_HANDSHAKE_CLIENT_KEY_EXCHANGE, optional=optional@entry=0, buf=buf@entry=0x7f29e8ffb830) at handshake.c:1465
#9 0x00007f2a005640cb in _gnutls_recv_client_kx_message (session=session@entry=0x7f29dc008060) at kx.c:563
#10 0x00007f2a00560572 in handshake_server (session=0x7f29dc008060) at handshake.c:3327
#11 gnutls_handshake (session=0x7f29dc008060) at handshake.c:2629
#12 0x00007f2a01ff2199 in tlsg_session_accept (session=0x7f29dc1133f0) at tls_g.c:363
#13 0x00007f2a01fefabb in ldap_pvt_tls_accept (sb=0x7f299c0051a0, ctx_arg=0x55d92cbca560) at tls2.c:425
Basically, what we see is one thread stuck in a busy loop doing read()s on the TCP socket which all return immediately with EAGAIN as the fd is in non-blocking mode.
In my cases, the client go offline just after sending the TLS client hello. That lasts for 15 minutes or about, probably until some timeout after which the TCP connection is eventually considered dead.
The important backtrace in there is the one from thread 11:
#0 0x00007fb288428474 in read () from /lib/x86_ 64-linux- gnu/libpthread. so.0 x86_64- linux-gnu/ liblber- 2.4.so. 2 x86_64- linux-gnu/ libgnutls. so.30 x86_64- linux-gnu/ libgnutls. so.30 x86_64- linux-gnu/ libgnutls. so.30 x86_64- linux-gnu/ libgnutls. so.30 x86_64- linux-gnu/ libgnutls. so.30 x86_64- linux-gnu/ libgnutls. so.30 x86_64- linux-gnu/ libldap_ r-2.4.so. 2 x86_64- linux-gnu/ libldap_ r-2.4.so. 2 ./../servers/ slapd/connectio n.c:1375
No symbol table info available.
#1 0x00007fb2890c4518 in ?? () from /usr/lib/
No symbol table info available.
#2 0x00007fb287895848 in ?? () from /usr/lib/
No symbol table info available.
#3 0x00007fb28788f96a in ?? () from /usr/lib/
No symbol table info available.
#4 0x00007fb287896d03 in ?? () from /usr/lib/
No symbol table info available.
#5 0x00007fb28789991c in ?? () from /usr/lib/
No symbol table info available.
#6 0x00007fb2878a10cb in ?? () from /usr/lib/
No symbol table info available.
#7 0x00007fb28789d572 in gnutls_handshake () from /usr/lib/
No symbol table info available.
#8 0x00007fb289304199 in ?? () from /usr/lib/
No symbol table info available.
#9 0x00007fb289301abb in ldap_pvt_tls_accept () from /usr/lib/
No symbol table info available.
#10 0x0000556b843e6f69 in connection_read (cri=<synthetic pointer>, s=430) at ../../.
debug symbols are missing there, but I have the exact same problem and get:
#0 0x00007f2a01101474 in __libc_read (fd=40, buf=0x7f29dc142ecb, nbytes=5) at ../sysdeps/ unix/sysv/ linux/read. c:27 0x7f29dc10e940, buf=0x7f29dc142ecb, len=5) at ../../. ./../libraries/ liblber/ sockbuf. c:829 0x7f2a01ff1da0 <tlsg_recv>, size=5, bufel=<synthetic pointer>, session= 0x7f29dc008060) at buffers.c:344 0x7f2a01ff1da0 <tlsg_recv>, size=5, bufel=<synthetic pointer>, session= 0x7f29dc008060) at buffers.c:424 io_read_ buffered (session= session@ entry=0x7f29dc0 08060, total=5, recv_type= recv_type@ entry=429496729 5, ms=0x7f29e8ffb41c) at buffers.c:579 0x7f29e8ffb470, htype=GNUTLS_ HANDSHAKE_ CLIENT_ KEY_EXCHANGE, type=GNUTLS_ HANDSHAKE, record_ params= 0x7f29dc1279f0, session= 0x7f29dc008060) at record.c:1045 recv_in_ buffers (session= session@ entry=0x7f29dc0 08060, type=type@ entry=GNUTLS_ HANDSHAKE, htype=htype@ entry=GNUTLS_ HANDSHAKE_ CLIENT_ KEY_EXCHANGE, ms=<optimized out>, ms@entry=0) at record.c:1173 handshake_ io_recv_ int (session= session@ entry=0x7f29dc0 08060, htype=htype@ entry=GNUTLS_ HANDSHAKE_ CLIENT_ KEY_EXCHANGE, hsk=hsk@ entry=0x7f29e8f fb580, optional= optional@ entry=0) at buffers.c:1412 recv_handshake (session= session@ entry=0x7f29dc0 08060, type=type@ entry=GNUTLS_ HANDSHAKE_ CLIENT_ KEY_EXCHANGE, optional= optional@ entry=0, buf=buf@ entry=0x7f29e8f fb830) at handshake.c:1465 recv_client_ kx_message (session= session@ entry=0x7f29dc0 08060) at kx.c:563 0x7f29dc008060) at handshake.c:3327 0x7f29dc008060) at handshake.c:2629 0x7f29dc1133f0) at tls_g.c:363 0x55d92cbca560) at tls2.c:425
#1 0x00007f2a01db0518 in sb_debug_read (sbiod=
#2 0x00007f2a00558848 in _gnutls_stream_read (ms=0x7f29e8ffb41c, pull_func=
#3 _gnutls_read (ms=0x7f29e8ffb41c, pull_func=
#4 _gnutls_
#5 0x00007f2a0055296a in recv_headers (ms=<optimized out>, record=
#6 _gnutls_
#7 0x00007f2a00559d03 in _gnutls_
#8 0x00007f2a0055c91c in _gnutls_
#9 0x00007f2a005640cb in _gnutls_
#10 0x00007f2a00560572 in handshake_server (session=
#11 gnutls_handshake (session=
#12 0x00007f2a01ff2199 in tlsg_session_accept (session=
#13 0x00007f2a01fefabb in ldap_pvt_tls_accept (sb=0x7f299c0051a0, ctx_arg=
and I've tracked it down to:
https:/ /bugs.openldap. org/show_ bug.cgi? id=8650# c12
Basically, what we see is one thread stuck in a busy loop doing read()s on the TCP socket which all return immediately with EAGAIN as the fd is in non-blocking mode.
In my cases, the client go offline just after sending the TLS client hello. That lasts for 15 minutes or about, probably until some timeout after which the TCP connection is eventually considered dead.
It can be reproduced by running on a client:
gdb --args ldapsearch -H ldaps:/ /ldap.example. com -x
Then in gdb:
break write
run
continue
Then the client is paused after sending the TLS "client hello".
https:/ /bugs.openldap. org/show_ bug.cgi? id=8650# c12 explains that it's https:/ /github. com/openldap/ openldap/ commit/ 7b5181da8cdd47a 13041f9ee36fa95 90a0fa6e48 that is responsible for the issue.
https:/ /github. com/openldap/ openldap/ commit/ 4c1ab16ade18a25 3dd81df7e6eced4 d920ac6a8e reverted that commit, but that one did not make it into bionic.
So cherry picking https:/ /github. com/openldap/ openldap/ commit/ 4c1ab16ade18a25 3dd81df7e6eced4 d920ac6a8e should fix it.