Utility user setup leaks memory | Valgrind: still reachable: malloc in acl_init_utility_user & init_dynamic_array2

[Laurynas Biveinis]

To repeat, run mysql-test-run percona_utility_user --valgrind-mysqld.

HEAP SUMMARY:
    in use at exit: 30,570,510 bytes in 218 blocks
  total heap usage: 44,831 allocs, 44,613 frees, 82,941,230 bytes allocated

25 bytes in 2 blocks are still reachable in loss record 1 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x90E832: my_strndup (my_malloc.c:155)
   by 0x58C4BD: acl_init_utility_user(char) (sql_acl.cc:1324)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x72558D: reload_acl_and_cache(THD*, unsigned long, TABLE_LIST*, int*) (sql_reload.cc:78)
   by 0x60A5FB: mysql_execute_command(THD*) (sql_parse.cc:3860)
   by 0x60F6DC: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:5867)
   by 0x602E75: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1062)
   by 0x602114: do_command(THD*) (sql_parse.cc:789)
   by 0x6F884D: do_handle_one_connection(THD*) (sql_connect.cc:1487)
   by 0x6F8342: handle_one_connection (sql_connect.cc:1402)
   by 0x9C2D5B: pfs_spawn_thread (pfs.cc:1015)
   by 0x545AE99: start_thread (pthread_create.c:308)
   by 0x619FCBC: clone (clone.S:112)

25 bytes in 2 blocks are indirectly lost in loss record 2 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x90E832: my_strndup (my_malloc.c:155)
   by 0x58C4BD: acl_init_utility_user(char) (sql_acl.cc:1324)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x58A05C: acl_init(bool) (sql_acl.cc:651)
   by 0x561C22: mysqld_main(int, char**) (mysqld.cc:4677)
   by 0x55A0DB: main (main.cc:25)

40 bytes in 1 blocks are still reachable in loss record 29 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x8E3D2C: init_dynamic_array2 (array.c:70)
   by 0x58C443: acl_init_utility_user(char) (sql_acl.cc:1314)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x72558D: reload_acl_and_cache(THD*, unsigned long, TABLE_LIST*, int*) (sql_reload.cc:78)
   by 0x60A5FB: mysql_execute_command(THD*) (sql_parse.cc:3860)
   by 0x60F6DC: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:5867)
   by 0x602E75: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1062)
   by 0x602114: do_command(THD*) (sql_parse.cc:789)
   by 0x6F884D: do_handle_one_connection(THD*) (sql_connect.cc:1487)
   by 0x6F8342: handle_one_connection (sql_connect.cc:1402)
   by 0x9C2D5B: pfs_spawn_thread (pfs.cc:1015)
   by 0x545AE99: start_thread (pthread_create.c:308)
   by 0x619FCBC: clone (clone.S:112)

49 bytes in 1 blocks are still reachable in loss record 49 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x58C090: acl_init_utility_user(char) (sql_acl.cc:1217)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x72558D: reload_acl_and_cache(THD*, unsigned long, TABLE_LIST*, int*) (sql_reload.cc:78)
   by 0x60A5FB: mysql_execute_command(THD*) (sql_parse.cc:3860)
   by 0x60F6DC: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:5867)
   by 0x602E75: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1062)
   by 0x602114: do_command(THD*) (sql_parse.cc:789)
   by 0x6F884D: do_handle_one_connection(THD*) (sql_connect.cc:1487)
   by 0x6F8342: handle_one_connection (sql_connect.cc:1402)
   by 0x9C2D5B: pfs_spawn_thread (pfs.cc:1015)
   by 0x545AE99: start_thread (pthread_create.c:308)
   by 0x619FCBC: clone (clone.S:112)

49 bytes in 1 blocks are definitely lost in loss record 50 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x58C090: acl_init_utility_user(char) (sql_acl.cc:1217)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x58A05C: acl_init(bool) (sql_acl.cc:651)
   by 0x561C22: mysqld_main(int, char**) (mysqld.cc:4677)
   by 0x55A0DB: main (main.cc:25)

61 bytes in 1 blocks are still reachable in loss record 51 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x58C0A3: acl_init_utility_user(char) (sql_acl.cc:1218)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x72558D: reload_acl_and_cache(THD*, unsigned long, TABLE_LIST*, int*) (sql_reload.cc:78)
   by 0x60A5FB: mysql_execute_command(THD*) (sql_parse.cc:3860)
   by 0x60F6DC: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:5867)
   by 0x602E75: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1062)
   by 0x602114: do_command(THD*) (sql_parse.cc:789)
   by 0x6F884D: do_handle_one_connection(THD*) (sql_connect.cc:1487)
   by 0x6F8342: handle_one_connection (sql_connect.cc:1402)
   by 0x9C2D5B: pfs_spawn_thread (pfs.cc:1015)
   by 0x545AE99: start_thread (pthread_create.c:308)
   by 0x619FCBC: clone (clone.S:112)

61 bytes in 1 blocks are definitely lost in loss record 52 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x58C0A3: acl_init_utility_user(char) (sql_acl.cc:1218)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x58A05C: acl_init(bool) (sql_acl.cc:651)
   by 0x561C22: mysqld_main(int, char**) (mysqld.cc:4677)
   by 0x55A0DB: main (main.cc:25)

65 (40 direct, 25 indirect) bytes in 1 blocks are definitely lost in loss record 59 of 99
   at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x90E42B: my_malloc (my_malloc.c:38)
   by 0x8E3D2C: init_dynamic_array2 (array.c:70)
   by 0x58C443: acl_init_utility_user(char) (sql_acl.cc:1314)
   by 0x58B286: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:974)
   by 0x58BDCC: acl_reload(THD*) (sql_acl.cc:1175)
   by 0x58A05C: acl_init(bool) (sql_acl.cc:651)
   by 0x561C22: mysqld_main(int, char**) (mysqld.cc:4677)
   by 0x55A0DB: main (main.cc:25)

Also seen at e.g.
http://jenkins.percona.com/job/percona-server-5.5-valgrind/183/BUILD_TYPE=valgrind,Host=ubuntu-precise-64bit/#showFailuresLink

[Roel Van de Paar (roel11)]

Seen in 5.6-QA also (malloc in acl_init_utility_user, init_dynamic_array2)

24 bytes in 1 blocks are still reachable in loss record 4 of 524
   at 0x4A069EE: malloc (vg_replace_malloc.c:270)
   by 0x997CA5: my_malloc (my_malloc.c:38)
   by 0x997E84: my_strndup (my_malloc.c:164)
   by 0x715803: acl_init_utility_user(char) (sql_acl.cc:1788)
   by 0x71602B: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:1297)
   by 0x716C4A: acl_reload(THD*) (sql_acl.cc:1624)
   by 0x71B305: acl_init(bool) (sql_acl.cc:1012)
   by 0x6348AE: mysqld_main(int, char**) (mysqld.cc:5573)
   by 0x3636E1ECDC: (below main) (in /lib64/libc-2.12.so)

40 bytes in 1 blocks are still reachable in loss record 183 of 524
   at 0x4A069EE: malloc (vg_replace_malloc.c:270)
   by 0x997CA5: my_malloc (my_malloc.c:38)
   by 0x97EEB4: init_dynamic_array2 (array.c:70)
   by 0x7157BA: acl_init_utility_user(char) (sql_acl.cc:1778)
   by 0x71602B: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:1297)
   by 0x716C4A: acl_reload(THD*) (sql_acl.cc:1624)
   by 0x71B305: acl_init(bool) (sql_acl.cc:1012)
   by 0x6348AE: mysqld_main(int, char**) (mysqld.cc:5573)
   by 0x3636E1ECDC: (below main) (in /lib64/libc-2.12.so)

49 bytes in 1 blocks are still reachable in loss record 236 of 524
   at 0x4A069EE: malloc (vg_replace_malloc.c:270)
   by 0x997CA5: my_malloc (my_malloc.c:38)
   by 0x7154A7: acl_init_utility_user(char) (sql_acl.cc:1664)
   by 0x71602B: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:1297)
   by 0x716C4A: acl_reload(THD*) (sql_acl.cc:1624)
   by 0x71B305: acl_init(bool) (sql_acl.cc:1012)
   by 0x6348AE: mysqld_main(int, char**) (mysqld.cc:5573)
   by 0x3636E1ECDC: (below main) (in /lib64/libc-2.12.so)

61 bytes in 1 blocks are still reachable in loss record 237 of 524
   at 0x4A069EE: malloc (vg_replace_malloc.c:270)
   by 0x997CA5: my_malloc (my_malloc.c:38)
   by 0x7154BA: acl_init_utility_user(char) (sql_acl.cc:1665)
   by 0x71602B: acl_load(THD*, TABLE_LIST*) (sql_acl.cc:1297)
   by 0x716C4A: acl_reload(THD*) (sql_acl.cc:1624)
   by 0x71B305: acl_init(bool) (sql_acl.cc:1012)
   by 0x6348AE: mysqld_main(int, char**) (mysqld.cc:5573)
   by 0x3636E1ECDC: (below main) (in /lib64/libc-2.12.so)

320 bytes in 1 blocks are still reachable in loss record 379 of 524
   at 0x4A069EE: malloc (vg_replace_malloc.c:270)
   by 0x997CA5: my_malloc (my_malloc.c:38)
   by 0x97EEB4: init_dynamic_array2 (array.c:70)
   by 0x9822BF: _my_hash_init (hash.c:92)
   by 0x9A8D9A: getopt_constraint_init(char) (my_getopt.cc:134)
   by 0x9A8DDF: getopt_constraint_find(char const*, unsigned long, char) (my_getopt.cc:152)
   by 0x9A8F84: getopt_constraint_get_min_value (my_getopt.cc:188)
   by 0x9A90EC: setval(my_option const*, void const*, char*, int) (my_getopt.cc:986)
   by 0x9A9E37: my_handle_options (my_getopt.cc:808)
   by 0x62F811: get_options(int*, char***) (mysqld.cc:8933)
   by 0x633177: init_common_variables() (mysqld.cc:3841)
   by 0x634635: mysqld_main(int, char**) (mysqld.cc:5383)
   by 0x3636E1ECDC: (below main) (in /lib64/libc-2.12.so)

[Roel Van de Paar (roel11)]

Does the "init_dynamic_array2" look a bit scary? i.e. is a global function of sorts failing ?

[Laurynas Biveinis (laurynas-biveinis)]

Not scary, it's a library function. The root cause is higher in the stack and it is localized.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-128