5.1 valgrind memory leaks in many tests

[Laurynas Biveinis]

Percona Server configured with debug and valgrind support (tests run with --innodb_use_purge_thread=32)

The leak stacktraces are the same (or at least very similar) for:
innodb_plugin.innodb_bug53674
innodb_plugin.innodb_bug30919
innodb_plugin.innodb_bug39438
innodb_plugin.innodb-use-sys-malloc
innodb_plugin.percona_bug692211
innodb_plugin.innodb-autoinc-56228
innodb_plugin.innodb_bug42101-nonzero
innodb_plugin.innodb_mysql_rbk
innodb_plugin.innodb_lock_wait_timeout_1
innodb_plugin.innodb_timeout_rollback
innodb_plugin.innodb-consistent innodb_plugin.innodb-semi-consistent
innodb_plugin.innodb_bug53756
innodb_plugin.innodb-autoinc-44030
innodb_plugin.innodb-autoinc-optimize
... and others

worker[1] Valgrind report from /home/laurynas/percona/src/percona-server/Percona-Server/mysql-test/var/1/log/mysqld.1.err after tests:
 innodb_plugin.innodb_bug53674
------------------------------------------------------------
HEAP SUMMARY:
    in use at exit: 3,053,344 bytes in 254 blocks
  total heap usage: 69,220 allocs, 68,966 frees, 65,297,285 bytes allocated

368 bytes in 1 blocks are definitely lost in loss record 1 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x8F4E82: mem_alloc_func (mem0mem.ic:529)
   by 0x94C613: trx_purge_sys_create (trx0purge.c:259)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

368 bytes in 1 blocks are definitely lost in loss record 2 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x8F4E82: mem_alloc_func (mem0mem.ic:529)
   by 0x94C644: trx_purge_sys_create (trx0purge.c:260)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

376 bytes in 1 blocks are possibly lost in loss record 3 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x92DC2A: row_purge_node_create (row0purge.c:76)
   by 0x94C3C2: trx_purge_graph_build (trx0purge.c:202)
   by 0x94C7AF: trx_purge_sys_create (trx0purge.c:273)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

632 bytes in 1 blocks are possibly lost in loss record 4 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x94C369: trx_purge_graph_build (trx0purge.c:196)
   by 0x94C7AF: trx_purge_sys_create (trx0purge.c:273)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

1,384 bytes in 1 blocks are possibly lost in loss record 5 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F5C9C: mem_heap_add_block (mem0mem.c:446)
   by 0x8F4671: mem_heap_alloc (mem0mem.ic:186)
   by 0x92DBF6: row_purge_node_create (row0purge.c:71)
   by 0x94C3C2: trx_purge_graph_build (trx0purge.c:202)
   by 0x94C7AF: trx_purge_sys_create (trx0purge.c:273)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)

4,960 bytes in 31 blocks are possibly lost in loss record 6 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x8F4E82: mem_alloc_func (mem0mem.ic:529)
   by 0x9611BA: sess_open (usr0sess.c:46)
   by 0x94C6CA: trx_purge_sys_create (trx0purge.c:267)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

5,704 bytes in 31 blocks are possibly lost in loss record 7 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x959292: trx_create (trx0trx.c:198)
   by 0x9611D5: sess_open (usr0sess.c:50)
   by 0x94C6CA: trx_purge_sys_create (trx0purge.c:267)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

11,280 bytes in 30 blocks are indirectly lost in loss record 8 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x92DC2A: row_purge_node_create (row0purge.c:76)
   by 0x94C3C2: trx_purge_graph_build (trx0purge.c:202)
   by 0x94C7AF: trx_purge_sys_create (trx0purge.c:273)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

11,656 bytes in 31 blocks are possibly lost in loss record 9 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x9590DB: trx_create (trx0trx.c:166)
   by 0x9611D5: sess_open (usr0sess.c:50)
   by 0x94C6CA: trx_purge_sys_create (trx0purge.c:267)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

11,656 bytes in 31 blocks are possibly lost in loss record 10 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x95919E: trx_create (trx0trx.c:178)
   by 0x9611D5: sess_open (usr0sess.c:50)
   by 0x94C6CA: trx_purge_sys_create (trx0purge.c:267)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

18,960 bytes in 30 blocks are indirectly lost in loss record 11 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x94C369: trx_purge_graph_build (trx0purge.c:196)
   by 0x94C7AF: trx_purge_sys_create (trx0purge.c:273)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

41,520 bytes in 30 blocks are indirectly lost in loss record 12 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F5C9C: mem_heap_add_block (mem0mem.c:446)
   by 0x8F4671: mem_heap_alloc (mem0mem.ic:186)
   by 0x92DBF6: row_purge_node_create (row0purge.c:71)
   by 0x94C3C2: trx_purge_graph_build (trx0purge.c:202)
   by 0x94C7AF: trx_purge_sys_create (trx0purge.c:273)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)

44,144 bytes in 31 blocks are possibly lost in loss record 13 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x8F4E82: mem_alloc_func (mem0mem.ic:529)
   by 0x958D5E: trx_create (trx0trx.c:94)
   by 0x9611D5: sess_open (usr0sess.c:50)
   by 0x94C6CA: trx_purge_sys_create (trx0purge.c:267)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)

72,128 (368 direct, 71,760 indirect) bytes in 1 blocks are definitely lost in loss record 14 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x8F6D51: mem_area_alloc (mem0pool.c:380)
   by 0x8F59C5: mem_heap_create_block (mem0mem.c:333)
   by 0x8F4D1D: mem_heap_create_func (mem0mem.ic:443)
   by 0x8F4E82: mem_alloc_func (mem0mem.ic:529)
   by 0x94C675: trx_purge_sys_create (trx0purge.c:261)
   by 0x957C82: trx_sys_init_at_db_start (trx0sys.c:1301)
   by 0x957D0A: trx_sys_create (trx0sys.c:1323)
   by 0x945D60: innobase_start_or_create_for_mysql (srv0start.c:1668)
   by 0x8C51A1: innobase_init(void*) (ha_innodb.cc:2524)
   by 0x7A976C: ha_initialize_handlerton(st_plugin_int*) (handler.cc:435)
   by 0x88C3B6: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1041)
   by 0x88CC39: plugin_init(int*, char**, int) (sql_plugin.cc:1268)
   by 0x643125: init_server_components() (mysqld.cc:4070)
   by 0x643EB5: main (mysqld.cc:4645)

770,048 bytes in 1 blocks are still reachable in loss record 15 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x9624F3: ut_malloc_low (ut0mem.c:106)
   by 0x962874: ut_malloc (ut0mem.c:244)
   by 0x8FD563: os_aio_simulated_handle (os0file.c:4203)
   by 0x9B17B6: fil_aio_wait (fil0fil.c:5110)
   by 0x943DA1: io_handler_thread (srv0start.c:474)
   by 0x4E35970: start_thread (pthread_create.c:304)
   by 0x13A356FF: ???

1,064,960 bytes in 1 blocks are still reachable in loss record 16 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x9624F3: ut_malloc_low (ut0mem.c:106)
   by 0x962874: ut_malloc (ut0mem.c:244)
   by 0x8FD563: os_aio_simulated_handle (os0file.c:4203)
   by 0x9B17B6: fil_aio_wait (fil0fil.c:5110)
   by 0x943DA1: io_handler_thread (srv0start.c:474)
   by 0x4E35970: start_thread (pthread_create.c:304)
   by 0x142366FF: ???

1,064,960 bytes in 1 blocks are still reachable in loss record 17 of 17
   at 0x4C2815C: malloc (vg_replace_malloc.c:236)
   by 0x9624F3: ut_malloc_low (ut0mem.c:106)
   by 0x962874: ut_malloc (ut0mem.c:244)
   by 0x8FD563: os_aio_simulated_handle (os0file.c:4203)
   by 0x9B17B6: fil_aio_wait (fil0fil.c:5110)
   by 0x943DA1: io_handler_thread (srv0start.c:474)
   by 0x4E35970: start_thread (pthread_create.c:304)
   by 0x152386FF: ???

LEAK SUMMARY:
   definitely lost: 1,104 bytes in 3 blocks
   indirectly lost: 71,760 bytes in 90 blocks
     possibly lost: 80,512 bytes in 158 blocks
   still reachable: 2,899,968 bytes in 3 blocks
        suppressed: 0 bytes in 0 blocks

For counts of detected and suppressed errors, rerun with: -v
ERROR SUMMARY: 11 errors from 11 contexts (suppressed: 4 from 4)

[Stewart Smith (stewart)]

We may already have some bugs open for this. would be good to verify on current percona server.

[Laurynas Biveinis (laurynas-biveinis)]

I have on my TODO list declaring multiple purge thread support stable, so I will re-check these failures once I get around doing that task.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1854