external (system) libevent cannot be used

[Arkadiusz Miśkiewicz]

Build system always uses internal copy of libevent (we all know bundled libs are root of evil ;).

Using always external is easy:
http://cvs.pld-linux.org/cgi-bin/cvsweb/packages/mariadb/mariadb-libevent.patch
but proper fix would be to allow to choose between internal and system one at configure time.

[Kristian Nielsen (knielsen)]

The problem here is that there is no way with ./configure to select the system libevent. This is needed for building .deb/.rpm for inclusion in distros and repositories.

Libevent should work like the other external dependencies (libz, libreadline, libssl) where one can at configure time select between bundled and system version.

[Kristian Nielsen (knielsen)]

User pallaert on IRC Freenode #maria reported an issue that shows one of the
problems with this.

He is using MariaDB with the Pinba plugin: http://pinba.org/

Pinba uses libevent internally.

In this case, mysqld hangs during shutdown. Here are relevant parts of a stack
trace when it hangs:

      Thread 5 (Thread 0x9488ab70 (LWP 18676)):
      #0 0xb78b1424 in __kernel_vsyscall ()
      #1 0x49858507 in syscall () from /lib/libc.so.6
      #2 0x0844cb6d in epoll_wait (epfd=15, events=0xaba2aaa8, maxevents=1023, timeout=-1) at epoll_sub.c:57
      #3 0x0844c637 in epoll_dispatch (base=0xaba2a8f8, arg=0xaba06828, tv=0x0) at epoll.c:195
      #4 0x08449cf0 in event_base_loop (base=0xaba2a8f8, flags=0) at event.c:500
      #5 0x08449a9d in event_base_dispatch (event_base=0xaba2a8f8) at event.c:386
      #6 0xb6068d8a in pinba_collector_main (arg=0x0) at main.cc:164
      #7 0x499168df in start_thread () from /lib/libpthread.so.0

      Thread 4 (Thread 0x94089b70 (LWP 18677)):
      #0 0xb78b1424 in __kernel_vsyscall ()
      #1 0x49815036 in nanosleep () from /lib/libc.so.6
      #2 0x49854b1d in usleep () from /lib/libc.so.6
      #3 0xb60672ae in pinba_stats_main (arg=0x0) at pool.cc:518
      #4 0x499168df in start_thread () from /lib/libpthread.so.0

      Thread 2 (Thread 0xb559eb70 (LWP 18690)):
      #0 0xb78b1424 in __kernel_vsyscall ()
      #1 0x49917f27 in pthread_join () from /lib/libpthread.so.0
      #2 0xb6056caf in pinba_engine_shutdown (p=0x0) at ha_pinba.cc:289
      #3 0x0837ad7f in ha_finalize_handlerton (plugin=0xaf25c70) at handler.cc:403
      #4 0x08433dc3 in plugin_deinitialize (plugin=0xaf25c70, ref_check=true) at sql_plugin.cc:828
      #5 0x08433feb in reap_plugins () at sql_plugin.cc:916
      #6 0x0843483d in plugin_shutdown () at sql_plugin.cc:1544
      #7 0x0824f5a5 in clean_up (print_message=true) at mysqld.cc:1328
      #8 0x082523b0 in unireg_end () at mysqld.cc:1254
      #9 0x08252ecb in kill_server (sig_ptr=0x0) at mysqld.cc:1189
      #10 0x08252ee2 in kill_server_thread (arg=0x938883a0) at mysqld.cc:1217
      #11 0x499168df in start_thread () from /lib/libpthread.so.0

      Thread 1 (Thread 0xb78906c0 (LWP 18653)):
      #0 0xb78b1424 in __kernel_vsyscall ()
      #1 0x4991b6e5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
      #2 0x087012f4 in safe_cond_wait (cond=0x88b2dc0, mp=0x88b2640, file=0x8767d80 "mysqld.cc", line=4667) at thr_mutex.c:496
      #3 0x0825232c in main (argc=9, argv=0xbfdc5cb4) at mysqld.cc:4667

Thread 2 is stuck in pinba_engine_shutdown(). This function does:

  pthread_cancel(collector_thread);
  pthread_join(collector_thread, NULL);

The collector_thread is Thread 5 in the stacktrace. It is stuck in epoll_wait
inside libevent.

Applying the patch above to use system libevent for mysqld solves the problem,
and there is no longer any hang during shutdown.

When mysqld is linked statically with the bundled libevent, and the pinba
plugin links to the system libevent.so (likely a different version), we get
inside the mysqld process a strange mix of code from the two different
libevent versions, which causes havoc of...

[Kristian Nielsen (knielsen)]

The fix for this will be in MariadB 5.1.43

[Kristian Nielsen (knielsen)]

Hm, it seems a complication here is that some of the distros we build for (like Ubuntu 8.04 hardy) has too old system libevent. At least the /usr/include/evutil.h is missing (it's included in eg. 9.10 karmic).

[Kristian Nielsen (knielsen)]

We can use system libevent from debian6 "squeeze" onwards and Ubuntu 9.10 "karmic" onward, but
on earlier releases we must use bundled, as system is too old.