InnoDB crash: Assertion failure... in file ../../storage/xtradb/include/buf0buf.ic line 544

[Casey Forbes]

Sorry that this isn't much of a report. Let me know if I can provide more info.

MariaDB 5.1.39 crashed - the error log reports:

100402 4:26:23 InnoDB: Assertion failure in thread 139685025216272 in file ../../storage/xtradb/include/buf0buf.ic line 544
InnoDB: Failing assertion: buf_page_in_file(bpage)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100402 4:26:23 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=1073741824
read_buffer_size=131072
max_used_connections=154
max_threads=202
threads_connected=74
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1903909 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x7f0adc40de30
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f0af44eed40 thread_stack 0x48000
[0x82cf44]
[0x4ad227]
[0x60b860]
[0x8bfc35]
[0x888691]
[0x7c9f6f]
[0x7b88e1]
[0x7b8e8a]
[0x77ce65]
[0x77f4c6]
[0x726319]
[0x57ba69]
[0x50bb81]
[0x50bf5f]
[0x51b4be]
[0x51ccf0]
[0x51d70c]
[0x4b814f]
[0x4ba821]
[0x4be39b]
[0x4c0303]
[0x4c0c86]
[0x4b3dbb]
[0x6077f0]
[0x8abe19]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x7f0ad43b3938 is an invalid pointer
thd->thread_id=1004063
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100402 04:26:27 mysqld_safe Number of processes running now: 0
100402 04:26:27 mysqld_safe mysqld restarted

[Sergey Petrunia (sergefp)]

Hello,

you're right - if we do not have a way to repeat the problem, it's difficult to diagnose/fix it. There are some crashing bugs already reported against xtradb but it is not possible to tell whether this is the same or different issue.

* could you please let us know what platform/package was used (or did you compile from source?). That information will allow to resolve the stack trace, which could give a clue.

* MariaDB 5.1.39 uses quite old xtradb 1.0.3-8. If you have a way to repeat the problem, could you try MariaDB 5.1.44? It could be that the issue was fixed.

[Sergey Petrunia (sergefp)]

Considering the above, changing status to incomplete (=don't have enough info to act on)

[Casey Forbes (caseyforbes)]

In case it helps, I compiled from the source on

   Linux 2.6.31-gentoo-r6 SMP x86_64

with these options:

--sysconfdir=/etc/mysql --enable-community-features --enable-assembler
--enable-profiling --with-charset=utf8
--with-extra-charsets=ascii,latin1
--with-plugins=partition,archive,heap,innobase,myisam,maria,pbxt,partition
--localstatedir=/var/lib/mysql --with-collation=utf8_general_ci
--with-server --with-mysqld-user=mysql
--with-unix-socket-path=/var/run/mysqld/mysqld.sock
--with-client-ldflags=-all-static --disable-shared --with-pic
--without-debug --with-extra-tools --with-geometry --with-readline
--with-row-based-replication --with-mysqld-ldflags=-all-static
--enable-thread-safe-client --enable-local-infile
--with-maria-tmp-tables --with-libevent

Casey

On Tue, 4 May 2010, Sergey Petrunia wrote:

> Hello,
>
> you're right - if we do not have a way to repeat the problem, it's
> difficult to diagnose/fix it. There are some crashing bugs already
> reported against xtradb but it is not possible to tell whether this is
> the same or different issue.
>
> * could you please let us know what platform/package was used (or did
> you compile from source?). That information will allow to resolve the
> stack trace, which could give a clue.
>
> * MariaDB 5.1.39 uses quite old xtradb 1.0.3-8. If you have a way to
> repeat the problem, could you try MariaDB 5.1.44? It could be that the
> issue was fixed.
>
> --
> InnoDB crash: Assertion failure... in file ../../storage/xtradb/include/buf0buf.ic line 544
> https://bugs.launchpad.net/bugs/553982
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Maria: Incomplete
>
> Bug description:
> Sorry that this isn't much of a report. Let me know if I can provide more info.
>
> MariaDB 5.1.39 crashed - the error log reports:
>
>
> 100402 4:26:23 InnoDB: Assertion failure in thread 139685025216272 in file ../../storage/xtradb/include/buf0buf.ic line 544
> InnoDB: Failing assertion: buf_page_in_file(bpage)
> InnoDB: We intentionally generate a memory trap.
> InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
> InnoDB: If you get repeated assertion failures or crashes, even
> InnoDB: immediately after the mysqld startup, there may be
> InnoDB: corruption in the InnoDB tablespace. Please refer to
> InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
> InnoDB: about forcing recovery.
> 100402 4:26:23 - mysqld got signal 6 ;
> This could be because you hit a bug. It is also possible that this binary
> or one of the libraries it was linked against is corrupt, improperly built,
> or misconfigured. This error can also be caused by malfunctioning hardware.
> We will try our best to scrape up some info that will hopefully help diagnose
> the problem, but since we have already crashed, something is definitely wrong
> and this may fail.
>
> key_buffer_size=1073741824
> read_buffer_size=131072
> max_used_connections=154
> max_threads=202
> threads_connected=74
> It is possible that mysqld could use up to
> key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1903909 K
> bytes of memory
> Hope that's ok; if not, ...

[Sergey Petrunia (sergefp)]

> In case it helps, I compiled from the source on
> Linux 2.6.31-gentoo-r6 SMP x86_64
<skip>

Thanks for the info. Alas, that won't be of much help because it doesn't allow to resolve the stack trace. (Is there anybody looking at this bug who could tell if the posted stack trace looks like real one at all? We see "thd->query at 0x7f0ad43b3938 is an invalid pointer" which could be an indication that stack has been damaged?)

Casey, does the problem continue to show up with any regularity?

[Casey Forbes (caseyforbes)]

No - it only happened once.

Is there anything I can/should do when I next rebuild Maria and
install a new version so that I can be more useful if I run into a
bug?

(Also, have I mentioned how much I appreciate the insanely fast InnoDB
recovery time in Maria/XtraDB/InnoDB plugin? Good work, whoever worked
on that)

Casey

On Tue, 11 May 2010, Sergey Petrunia wrote:

>
>> In case it helps, I compiled from the source on
>> Linux 2.6.31-gentoo-r6 SMP x86_64
> <skip>
>
> Thanks for the info. Alas, that won't be of much help because it
> doesn't allow to resolve the stack trace. (Is there anybody looking at
> this bug who could tell if the posted stack trace looks like real one at
> all? We see "thd->query at 0x7f0ad43b3938 is an invalid pointer" which
> could be an indication that stack has been damaged?)
>
> Casey, does the problem continue to show up with any regularity?
>
> --
> InnoDB crash: Assertion failure... in file ../../storage/xtradb/include/buf0buf.ic line 544
> https://bugs.launchpad.net/bugs/553982
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Maria: Incomplete
>
> Bug description:
> Sorry that this isn't much of a report. Let me know if I can provide more info.
>
> MariaDB 5.1.39 crashed - the error log reports:
>
>
> 100402 4:26:23 InnoDB: Assertion failure in thread 139685025216272 in file ../../storage/xtradb/include/buf0buf.ic line 544
> InnoDB: Failing assertion: buf_page_in_file(bpage)
> InnoDB: We intentionally generate a memory trap.
> InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
> InnoDB: If you get repeated assertion failures or crashes, even
> InnoDB: immediately after the mysqld startup, there may be
> InnoDB: corruption in the InnoDB tablespace. Please refer to
> InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
> InnoDB: about forcing recovery.
> 100402 4:26:23 - mysqld got signal 6 ;
> This could be because you hit a bug. It is also possible that this binary
> or one of the libraries it was linked against is corrupt, improperly built,
> or misconfigured. This error can also be caused by malfunctioning hardware.
> We will try our best to scrape up some info that will hopefully help diagnose
> the problem, but since we have already crashed, something is definitely wrong
> and this may fail.
>
> key_buffer_size=1073741824
> read_buffer_size=131072
> max_used_connections=154
> max_threads=202
> threads_connected=74
> It is possible that mysqld could use up to
> key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1903909 K
> bytes of memory
> Hope that's ok; if not, decrease some variables in the equation.
>
> thd: 0x7f0adc40de30
> Attempting backtrace. You can use the following information to find out
> where mysqld died. If you see no messages after this, something went
> terribly wrong...
> stack_bottom = 0x7f0af44eed40 thread_stack 0x48000
> [0x82cf44]
> [0x4ad227]
> [0x60b860]
> [0x8bfc35]
> [0x888691]
> [0x7c9f6f]
> [0x7b88e1]
> [0x7b8e8a]
> [0x77ce65]
> [0x77f4c6]
> [0x726319]
> [0x57ba69]
> [0x50bb81]
> [0x50bf5f]
> [0x51b4be]
> [0x51ccf0]
> [0x51d70c]
> [0x4b814f]
> [0x4ba821]
> [0x4be39b]
>...