Uninitialised memory write in XTDatabaseLog::xlog_append

Bug #451080 reported by Oleksandr "Sanja" Byelkin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MariaDB
Fix Released
Medium
Hakan Küçükyılmaz
PBXT
Fix Committed
Undecided
Vladimir Kolesnikov
maria (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

valgrind catch uninitialised memory write in pbxt.alias (for example, the error mentioned in many other cases), stack is following:

==9024== Thread 4:
==9024== Syscall param pwrite64(buf) points to uninitialised byte(s)
==9024== at 0x504F3A8: (within /lib64/libpthread-2.9.so)
==9024== by 0xA0E46C: xt_pwrite_file(XTOpenFile*, long, unsigned long, void*, XTIOStats*, XTThread*) (filesys_xt.cc:848)
==9024== by 0x9F140C: XTDatabaseLog::xlog_append(XTThread*, unsigned long, unsigned char*, unsigned long, unsigned char*, int, unsigned int*, long*) (xactlog_xt.cc:1111)
==9024== by 0x9F21D4: xt_xlog_log_data(XTThread*, unsigned long, XTXactLogBuffer*, int) (xactlog_xt.cc:1487)
==9024== by 0x9EA751: xt_xn_log_tab_id(XTThread*, unsigned int) (xaction_xt.cc:1471)
==9024== by 0x9DBF12: xt_create_table(XTThread*, XTPathStr*, XTDictionary*) (table_xt.cc:1502)
==9024== by 0x9AB32A: ha_pbxt::create(char const*, st_table*, st_ha_create_information*) (ha_pbxt.cc:5086)
==9024== by 0x7A4B26: handler::ha_create(char const*, st_table*, st_ha_create_information*) (handler.cc:3376)
==9024== by 0x7A7C19: ha_create_table(THD*, char const*, char const*, char const*, st_ha_create_information*, bool) (handler.cc:3587)
==9024== by 0x75875B: rea_create_table(THD*, char const*, char const*, char const*, st_ha_create_information*, List<Create_field>&, unsigned int, st_key*, handler*) (unireg.cc:416)
==9024== by 0x7C61BE: mysql_create_table_no_lock(THD*, char const*, char const*, st_ha_create_information*, Alter_info*, bool, unsigned int) (sql_table.cc:3853)
==9024== by 0x7C658F: mysql_create_table(THD*, char const*, char const*, st_ha_create_information*, Alter_info*, bool, unsigned int) (sql_table.cc:3960)
==9024== by 0x67C4AA: mysql_execute_command(THD*) (sql_parse.cc:2732)
==9024== by 0x683ECE: mysql_parse(THD*, char const*, unsigned int, char const**) (sql_parse.cc:5979)
==9024== by 0x684CD8: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1223)
==9024== by 0x68602C: do_command(THD*) (sql_parse.cc:862)
==9024== Address 0xf096292 is 50 bytes inside a block of size 1,049,088 alloc'd
==9024== at 0x4C24CFE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==9024== by 0x9B9A51: xt_malloc(XTThread*, unsigned long) (memory_xt.cc:101)
==9024== by 0x9F4055: XTDatabaseLog::xlog_setup(XTThread*, XTDatabase*, long, unsigned long, int) (xactlog_xt.cc:639)
==9024== by 0x9EB76C: xt_xn_init_db(XTThread*, XTDatabase*) (xaction_xt.cc:1103)
==9024== by 0x9FD727: xt_get_database(XTThread*, char*, int) (database_xt.cc:469)
==9024== by 0x9FD96A: xt_open_database(XTThread*, char*, int) (database_xt.cc:625)
==9024== by 0x9C362A: xn_xres_run_recovery_thread(XTThread*) (restart_xt.cc:3206)
==9024== by 0x9E32A0: thr_main (thread_xt.cc:1022)
==9024== by 0x5048016: start_thread (in /lib64/libpthread-2.9.so)
==9024== by 0x602248C: clone (in /lib64/libc-2.9.so)
==9024==
==9024== Syscall param pwrite64(buf) points to uninitialised byte(s)
==9024== at 0x504F3A8: (within /lib64/libpthread-2.9.so)
==9024== by 0xA0E46C: xt_pwrite_file(XTOpenFile*, long, unsigned long, void*, XTIOStats*, XTThread*) (filesys_xt.cc:848)
==9024== by 0x9F140C: XTDatabaseLog::xlog_append(XTThread*, unsigned long, unsigned char*, unsigned long, unsigned char*, int, unsigned int*, long*) (xactlog_xt.cc:1111)
==9024== by 0x9F223E: XTDatabaseLog::xlog_flush(XTThread*) (xactlog_xt.cc:729)
==9024== by 0x9D5002: xt_sync_flush_table(XTThread*, XTOpenTable*) (table_xt.cc:2170)
==9024== by 0x9FC4C7: db_lock_table_pool(XTThread*, XTDatabase*, unsigned int, int, int) (database_xt.cc:830)
==9024== by 0x9FC8E1: xt_db_lock_table_pool_by_name(XTThread*, XTDatabase*, XTPathStr*, int, int, int, int, XTTable**) (database_xt.cc:901)
==9024== by 0x9D62F8: tab_lock_table(XTThread*, XTPathStr*, int, int, int, XTTable**) (table_xt.cc:1259)
==9024== by 0x9D7E73: xt_drop_table(XTThread*, XTPathStr*, int) (table_xt.cc:1627)
==9024== by 0x9AE8A7: ha_pbxt::delete_table(char const*) (ha_pbxt.cc:4759)
==9024== by 0x7A4B92: handler::ha_delete_table(char const*) (handler.cc:3346)
==9024== by 0x7AA132: ha_delete_table(THD*, handlerton*, char const*, char const*, char const*, bool) (handler.cc:1966)
==9024== by 0x7CA94A: mysql_rm_table_part2(THD*, TABLE_LIST*, bool, bool, bool, bool) (sql_table.cc:1976)
==9024== by 0x7CAE68: mysql_rm_table(THD*, TABLE_LIST*, char, char) (sql_table.cc:1749)
==9024== by 0x67E890: mysql_execute_command(THD*) (sql_parse.cc:3398)
==9024== by 0x683ECE: mysql_parse(THD*, char const*, unsigned int, char const**) (sql_parse.cc:5979)
==9024== Address 0xf096414 is 436 bytes inside a block of size 1,049,088 alloc'd
==9024== at 0x4C24CFE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==9024== by 0x9B9A51: xt_malloc(XTThread*, unsigned long) (memory_xt.cc:101)
==9024== by 0x9F4055: XTDatabaseLog::xlog_setup(XTThread*, XTDatabase*, long, unsigned long, int) (xactlog_xt.cc:639)
==9024== by 0x9EB76C: xt_xn_init_db(XTThread*, XTDatabase*) (xaction_xt.cc:1103)
==9024== by 0x9FD727: xt_get_database(XTThread*, char*, int) (database_xt.cc:469)
==9024== by 0x9FD96A: xt_open_database(XTThread*, char*, int) (database_xt.cc:625)
==9024== by 0x9C362A: xn_xres_run_recovery_thread(XTThread*) (restart_xt.cc:3206)
==9024== by 0x9E32A0: thr_main (thread_xt.cc:1022)
==9024== by 0x5048016: start_thread (in /lib64/libpthread-2.9.so)
==9024== by 0x602248C: clone (in /lib64/libc-2.9.so)

Many other cases can be found in:
http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.1
http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.2
http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.3
http://askmonty.org/buildbot/builders/gentoo-amd64-sanja/builds/4/steps/test_1/logs/mysqld.1.err.4

Can be repeated if run pbxt test suite under valgrind (valgrind build (one of BUILD/compile*valgrind* ) and --valgrind parameter of mysql-test-run)

Related branches

Changed in maria (Ubuntu):
status: New → Invalid
Changed in pbxt:
status: New → In Progress
assignee: nobody → Vladimir Kolesnikov (vkolesnikov)
Revision history for this message
Vladimir Kolesnikov (vkolesnikov) wrote :

Hi Sanja,

I have seen this case before and my analysis showed that the uninitialized bytes are at the tail of 512-byte blocks. This is not a bug. This happens because we write data block-by-block and the last block is likely to be only partially filled with data. If you have evidence of any real probem with this please let me know, otherwise I will close the bug.

BR,
Vladimir

Revision history for this message
Mark Callaghan (mdcallag) wrote :

Other code in MySQL avoids warnings like this by fully writing blocks when HAVE_purify is defined. Can the same be done here? I prefer that approach over adding more valgrind suppressions.

Revision history for this message
Vladimir Kolesnikov (vkolesnikov) wrote :

Mark,

I think this is possible... Thanks for pointing the right macro to use...

Revision history for this message
Michael Widenius (monty) wrote : [Bug 451080] Re: Uninitialised memory write in XTDatabaseLog::xlog_append

Hi!

>>>>> "Vladimir" == Vladimir Kolesnikov <email address hidden> writes:

Vladimir> Hi Sanja,
Vladimir> I have seen this case before and my analysis showed that the
Vladimir> uninitialized bytes are at the tail of 512-byte blocks. This is not a
Vladimir> bug. This happens because we write data block-by-block and the last
Vladimir> block is likely to be only partially filled with data. If you have
Vladimir> evidence of any real probem with this please let me know, otherwise I
Vladimir> will close the bug.

We would prefer to not have this happen when compiled with HAVE_valgrind.
For example, if we ignore these, we will also miss bugs when you
compile uninitialized bytes into the middle of a block.

For example, in MyISAM we handle it when writing a block:

#ifdef HAVE_valgrind
  {
    length=mi_getint(buff);
    bzero((uchar*) buff+length,keyinfo->block_length-length);
    length=keyinfo->block_length;
  }
#endif

In other words, we explicitly fill the not used parts with 0 to ensure
that valgrind doesn't complain.

HAVE_valgrind is only defined in debug builds, so there is no speed
penally for production systems.

Regards,
Monty

Revision history for this message
Vladimir Kolesnikov (vkolesnikov) wrote :

Hi Monty,

thanks for the input. I have added the code, it solves the problem. Will push it tomorrow.

Changed in pbxt:
status: In Progress → Fix Committed
Revision history for this message
Hakan Küçükyılmaz (hakan-askmonty) wrote :

Vladimir, I guess this fix is already released. Can you please point me to the bzr patch, so I can check it in our sources?

Thanks

Changed in maria:
assignee: nobody → Hakan Küçükyılmaz (hakan-askmonty)
status: New → Fix Committed
Revision history for this message
Vladimir Kolesnikov (vkolesnikov) wrote :

hi Hakan!

the fix is here:

lp:~vkolesnikov/pbxt/pbxt-bug-451080

revisions 718 and 719.

BR

Changed in maria:
importance: Undecided → Medium
affects: maria (Ubuntu) → ubuntu
affects: ubuntu → maria (Ubuntu)
Changed in maria:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers