Comment 3 for bug 1710278

Revision history for this message
Mike Pontillo (mpontillo) wrote :

Assuming the debug symbols I grabbed[1] for my install of bind9 on Xenial match yours (I have bind9 version 1:9.10.3.dfsg.P4-8ubuntu1.7 installed per "apt-cache policy bind9"), I did the following to grab a traceback:

$ sudo apt-get install bind9-dbgsym libdns162-dbgsym libisc160-dbgsym
$ gdb /usr/sbin/named core
(gdb) set pagination off
(gdb) thread apply all bt
... [2] ...

Looking at the backtrace in [2], the interesting parts to me are threads 8, 11 and 20, which are possibly involved in a deadlock[3]. Looks like one of the threads is reloading the configuration (something we would expect MAAS to do), and the other is calling dns_resolver_shutdown() via view_flushanddetach().

[1]: https://wiki.ubuntu.com/Debug%20Symbol%20Packages
[2]: http://paste.ubuntu.com/25292729/
[3]:
Thread 8 (Thread 0x7f95226aa700 (LWP 3203)):
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1 0x00007f952a351efe in __GI___pthread_mutex_lock (mutex=mutex@entry=0x7f94d4014fe8) at ../nptl/pthread_mutex_lock.c:135
#2 0x00007f952b7a0794 in dns_view_weakdetach (viewp=viewp@entry=0x7f9504389780) at ../../../lib/dns/view.c:597
#3 0x00007f952b7993de in destroy (val=0x7f9504389750) at ../../../lib/dns/validator.c:3891
#4 0x00007f952b79927b in dns_validator_destroy (validatorp=validatorp@entry=0x7f9519462628) at ../../../lib/dns/validator.c:3915
#5 0x00007f952b76b9d1 in validated (task=<optimized out>, event=0x7f95194625d0) at ../../../lib/dns/resolver.c:4722
#6 0x00007f952a9a6360 in dispatch (manager=0x7f952be3b010) at ../../../lib/isc/task.c:1130
#7 run (uap=0x7f952be3b010) at ../../../lib/isc/task.c:1302
#8 0x00007f952a34f6ba in start_thread (arg=0x7f95226aa700) at pthread_create.c:333
#9 0x00007f9529a993dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 11 (Thread 0x7f9520ea7700 (LWP 3206)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f952a9a516b in isc__task_beginexclusive (task0=<optimized out>) at ../../../lib/isc/task.c:1717
#2 0x0000557c34997dc1 in load_configuration (filename=<optimized out>, server=server@entry=0x7f952be44010, first_time=first_time@entry=isc_boolean_false) at ../../../bin/named/server.c:5651
#3 0x0000557c3499a826 in loadconfig (server=0x7f952be44010) at ../../../bin/named/server.c:7162
#4 0x0000557c3499ad48 in reload (server=0x7f952be44010) at ../../../bin/named/server.c:7183
#5 ns_server_reloadcommand (server=0x7f952be44010, args=args@entry=0x7f94fc120af0 "reload", text=text@entry=0x7f9520ea6590) at ../../../bin/named/server.c:7416
#6 0x0000557c34975db5 in ns_control_docommand (message=<optimized out>, text=text@entry=0x7f9520ea6590) at ../../../bin/named/control.c:102
#7 0x0000557c34978b97 in control_recvmessage (task=0x7f952be51010, event=<optimized out>) at ../../../bin/named/controlconf.c:458
#8 0x00007f952a9a6360 in dispatch (manager=0x7f952be3b010) at ../../../lib/isc/task.c:1130
#9 run (uap=0x7f952be3b010) at ../../../lib/isc/task.c:1302
#10 0x00007f952a34f6ba in start_thread (arg=0x7f9520ea7700) at pthread_create.c:333
#11 0x00007f9529a993dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 20 (Thread 0x7f951c69e700 (LWP 3215)):
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1 0x00007f952a351efe in __GI___pthread_mutex_lock (mutex=0x7f94d43429f8) at ../nptl/pthread_mutex_lock.c:135
#2 0x00007f952b7642a7 in dns_resolver_shutdown (res=0x7f952be56690) at ../../../lib/dns/resolver.c:9035
#3 0x00007f952b7a06a1 in view_flushanddetach (viewp=viewp@entry=0x7f94fc02cc30, flush=flush@entry=isc_boolean_false) at ../../../lib/dns/view.c:508
#4 0x00007f952b7a0757 in dns_view_detach (viewp=viewp@entry=0x7f94fc02cc30) at ../../../lib/dns/view.c:557
#5 0x0000557c3496e55b in ns_client_endrequest (client=0x7f94fc02cbe0) at ../../../bin/named/client.c:694
#6 exit_check (client=0x7f94fc02cbe0) at ../../../bin/named/client.c:382
#7 0x0000557c34970150 in ns_client_detach (clientp=clientp@entry=0x7f951c69ce18) at ../../../bin/named/client.c:2833
#8 0x0000557c34989ee2 in query_find (client=0x0, event=0x0, event@entry=0x7f9518a862b0, qtype=<optimized out>, qtype@entry=0) at ../../../bin/named/query.c:8328
#9 0x0000557c349921cf in query_resume (task=<optimized out>, event=0x7f9518a862b0) at ../../../bin/named/query.c:3826
#10 0x00007f952a9a6360 in dispatch (manager=0x7f952be3b010) at ../../../lib/isc/task.c:1130
#11 run (uap=0x7f952be3b010) at ../../../lib/isc/task.c:1302
#12 0x00007f952a34f6ba in start_thread (arg=0x7f951c69e700) at pthread_create.c:333
#13 0x00007f9529a993dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109