For clarity, the only bug that has been established with 100% certainty so far is this one in Log_event::read_log_event:
==13638== 1,068 (256 direct, 812 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 8
==13638== at 0x4A078B8: malloc (vg_replace_malloc.c:270)
==13638== by 0x7A3361: my_malloc (in /data/chroot_dc_head_new/usr/sbin/mysqld)
==13638== by 0x724D7C: Log_event::read_log_event(char const*, unsigned int, char const**, Format_description_log_event const*) (in /data/chroot_dc_head_new/usr/sbin/mysqld)
==13638== by 0x72856B: Log_event::read_log_event(st_io_cache*, st_mysql_mutex*, Format_description_log_event const*) (in /data/chroot_dc_head_new/usr/sbin/mysqld)
==13638== by 0x52FA09: ??? (in /data/chroot_dc_head_new/usr/sbin/mysqld)
==13638== by 0x53104D: handle_slave_sql (in /data/chroot_dc_head_new/usr/sbin/mysqld)
==13638== by 0x3DC3E0673C: start_thread (in /lib64/libpthread-2.5.so)
==13638== by 0x3DC2ED3D1C: clone (in /lib64/libc-2.5.so)
==16132== 256 bytes in 1 blocks are definitely lost in loss record 4 of 8
==16132== at 0x4A07848: malloc (vg_replace_malloc.c:270)
==16132== by 0x7D1FE1: my_malloc (in /usr/local/percona/mysql-5.5.25a/usr/sbin/mysqld)
==16132== by 0x74F6E3: Log_event::read_log_event(char const*, unsigned int, char const**, Format_description_log_event const*) (in /usr/local/percona/mysql-5.5.25a/usr/sbin/mysqld)
==16132== by 0x752558: Log_event::read_log_event(st_io_cache*, st_mysql_mutex*, Format_description_log_event const*) (in /usr/local/percona/mysql-5.5.25a/usr/sbin/mysqld)
==16132== by 0x518993: ??? (in /usr/local/percona/mysql-5.5.25a/usr/sbin/mysqld)
==16132== by 0x52185C: ??? (in /usr/local/percona/mysql-5.5.25a/usr/sbin/mysqld)
==16132== by 0x522D33: handle_slave_sql (in /usr/local/percona/mysql-5.5.25a/usr/sbin/mysqld)
==16132== by 0x3ECD60673C: start_thread (in /lib64/libpthread-2.5.so)
==16132== by 0x3ECC6D456C: clone (in /lib64/libc-2.5.so)
Though 256 bytes may not look big, in the same trace there is "possibly lost: 315,964 bytes in 2 blocks" and most of those possibly lost bytes *are also* from Log_event::read_log_event. And, this was only a 4 hour run. It is also likely that only certain field types and/or certain queries etc. cause loss, so it may not be possible to extrapolate numbers, or even reliably produce a Valgrind stack on each run.
I believe both these stacks were seen in connection with replication traffic only? It would be interesting also to see the my.cnf files used by Aleksandr and Alex when these particular issues were observed to see if there are any similarities.
It would further be good to confirm if the issue also shows in the latest 5.5 MySQL. If so, we will report an upstream bug. Aleksandr, Alex, are you at liberty to test this?
For clarity, the only bug that has been established with 100% certainty so far is this one in Log_event: :read_log_ event:
==13638== 1,068 (256 direct, 812 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 8 malloc. c:270) dc_head_ new/usr/ sbin/mysqld) :read_log_ event(char const*, unsigned int, char const**, Format_ description_ log_event const*) (in /data/chroot_ dc_head_ new/usr/ sbin/mysqld) :read_log_ event(st_ io_cache* , st_mysql_mutex*, Format_ description_ log_event const*) (in /data/chroot_ dc_head_ new/usr/ sbin/mysqld) dc_head_ new/usr/ sbin/mysqld) dc_head_ new/usr/ sbin/mysqld) libpthread- 2.5.so)
==13638== at 0x4A078B8: malloc (vg_replace_
==13638== by 0x7A3361: my_malloc (in /data/chroot_
==13638== by 0x724D7C: Log_event:
==13638== by 0x72856B: Log_event:
==13638== by 0x52FA09: ??? (in /data/chroot_
==13638== by 0x53104D: handle_slave_sql (in /data/chroot_
==13638== by 0x3DC3E0673C: start_thread (in /lib64/
==13638== by 0x3DC2ED3D1C: clone (in /lib64/libc-2.5.so)
(From https:/ /launchpadlibra rian.net/ 118121708/ valgrind_ log.p13638. 20120928. log - From Aleksandr - Note #11)
And another occurrence of the same:
==16132== 256 bytes in 1 blocks are definitely lost in loss record 4 of 8 malloc. c:270) percona/ mysql-5. 5.25a/usr/ sbin/mysqld) :read_log_ event(char const*, unsigned int, char const**, Format_ description_ log_event const*) (in /usr/local/ percona/ mysql-5. 5.25a/usr/ sbin/mysqld) :read_log_ event(st_ io_cache* , st_mysql_mutex*, Format_ description_ log_event const*) (in /usr/local/ percona/ mysql-5. 5.25a/usr/ sbin/mysqld) percona/ mysql-5. 5.25a/usr/ sbin/mysqld) percona/ mysql-5. 5.25a/usr/ sbin/mysqld) percona/ mysql-5. 5.25a/usr/ sbin/mysqld) libpthread- 2.5.so)
==16132== at 0x4A07848: malloc (vg_replace_
==16132== by 0x7D1FE1: my_malloc (in /usr/local/
==16132== by 0x74F6E3: Log_event:
==16132== by 0x752558: Log_event:
==16132== by 0x518993: ??? (in /usr/local/
==16132== by 0x52185C: ??? (in /usr/local/
==16132== by 0x522D33: handle_slave_sql (in /usr/local/
==16132== by 0x3ECD60673C: start_thread (in /lib64/
==16132== by 0x3ECC6D456C: clone (in /lib64/libc-2.5.so)
(From https:/ /launchpadlibra rian.net/ 114133058/ percona_ 5.5.25a_ valgrind_ leak_summary_ 2012-08- 31 - Alex - Note #5)
Though 256 bytes may not look big, in the same trace there is "possibly lost: 315,964 bytes in 2 blocks" and most of those possibly lost bytes *are also* from Log_event: :read_log_ event. And, this was only a 4 hour run. It is also likely that only certain field types and/or certain queries etc. cause loss, so it may not be possible to extrapolate numbers, or even reliably produce a Valgrind stack on each run.
I believe both these stacks were seen in connection with replication traffic only? It would be interesting also to see the my.cnf files used by Aleksandr and Alex when these particular issues were observed to see if there are any similarities.
It would further be good to confirm if the issue also shows in the latest 5.5 MySQL. If so, we will report an upstream bug. Aleksandr, Alex, are you at liberty to test this?