Bug #1188297 “mysqld crashes on startup” : Bugs : Percona Server moved to https://jira.percona.com/projects/PS

Revision history for this message

Valerii Kravchuk (valerii-kravchuk) wrote on 2013-06-06:

#1

What exact version of Percona Server do you use?

Please, send the entire error log from this server and your my.cnf file content.

Changed in percona-server:
status:	New → Incomplete

Revision history for this message

Artūras Šlajus (x11-arturaz) wrote on 2013-06-08:

#2

Download full text (5.6 KiB)

# mysqld --version
mysqld Ver 5.5.31-30.3-log for Linux on i686 (Percona Server (GPL), Release 30.3)

# apt-cache show percona-server-server
Package: percona-server-server
Source: percona-server-5.5
Version: 5.5.31-rel30.3-520.lucid
Architecture: i386
Maintainer: Percona Server Development Team <email address hidden>
Installed-Size: 8
Depends: percona-server-server-5.5
Homepage: http://www.percona.com/software/percona-server/
Priority: extra
Section: database
Filename: pool/main/p/percona-server-5.5/percona-server-server_5.5.31-rel30.3-520.lucid_i386.deb
Size: 2744
SHA256: ca780d0e09427ffc1bb96814fb83f4f85d378b9053a93154c5f56cab13433385
SHA1: 32fe45ab1d0aaa440de693ff60b48ab3acce2d3c
MD5sum: 81badbfebbb0fd733a1339390abd862a

# cat /etc/mysql/my.cnf

[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock

[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0

[mysqld]
user = mysql
socket = /var/run/mysqld/mysqld.sock
pid-file = /var/run/mysqld/mysqld.pid
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
skip-external-locking
bind-address = 127.0.0.1
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
myisam-recover = BACKUP
query_cache_limit = 1M
query_cache_size = 16M

log_error = /var/log/mysql/error.log

slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 0.1
expire_logs_days = 10
max_binlog_size = 100M

[mysqldump]
quick
quote-names
max_allowed_packet = 16M

[isamchk]
key_buffer = 16M

!includedir /etc/mysql/conf.d/

# cat /etc/mysql/conf.d/*

[mysqld]
innodb_file_per_table
innodb_rollback_on_timeout = ON
innodb_lock_wait_timeout = 10
# http://dev.mysql.com/doc/refman/5.5/en/innodb-auto-increment-handling.html#innodb-auto-increment-configurable
# This is needed to allow multiple LOAD DATA statements to execute parallely.
innodb_autoinc_lock_mode = 2
# To allow replication on lock mode 2.
binlog_format=row

That's pretty much entire error log there after server restart...

# > /var/log/mysql/error.log
# /etc/init.d/mysql start
* Starting MySQL (Percona Server) database server mysqld [fail]
# cat /var/log/mysql/error.log
130608 07:50:46 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
130608 7:50:46 [Warning] You need to use --log-bin to make --binlog-format work.
130608 7:50:46 [Note] Plugin 'FEDERATED' is disabled.
130608 7:50:46 InnoDB: The InnoDB memory heap is disabled
130608 7:50:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130608 7:50:46 InnoDB: Compressed tables use zlib 1.2.3
130608 7:50:46 InnoDB: Using Linux native AIO
130608 7:50:46 InnoDB: Initializing buffer pool, size = 128.0M
130608 7:50:46 InnoDB: Completed initialization of buffer pool
130608 7:50:46 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 668771298910
130608 7:50:46 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewri...

# mysqld --version
mysqld  Ver 5.5.31-30.3-log for Linux on i686 (Percona Server (GPL), Release 30.3)

# apt-cache show percona-server-server
Package: percona-server-server
Source: percona-server-5.5
Version: 5.5.31-rel30.3-520.lucid
Architecture: i386
Maintainer: Percona Server Development Team <mysql-dev@percona.com>
Installed-Size: 8
Depends: percona-server-server-5.5
Homepage: http://www.percona.com/software/percona-server/
Priority: extra
Section: database
Filename: pool/main/p/percona-server-5.5/percona-server-server_5.5.31-rel30.3-520.lucid_i386.deb
Size: 2744
SHA256: ca780d0e09427ffc1bb96814fb83f4f85d378b9053a93154c5f56cab13433385
SHA1: 32fe45ab1d0aaa440de693ff60b48ab3acce2d3c
MD5sum: 81badbfebbb0fd733a1339390abd862a

# cat /etc/mysql/my.cnf

[client]
port    = 3306
socket    = /var/run/mysqld/mysqld.sock

[mysqld_safe]
socket    = /var/run/mysqld/mysqld.sock
nice    = 0

[mysqld]
user    = mysql
socket    = /var/run/mysqld/mysqld.sock
pid-file    = /var/run/mysqld/mysqld.pid
port    = 3306
basedir   = /usr
datadir   = /var/lib/mysql
tmpdir    = /tmp
skip-external-locking
bind-address    = 127.0.0.1
key_buffer    = 16M
max_allowed_packet  = 16M
thread_stack    = 192K
thread_cache_size       = 8
myisam-recover         = BACKUP
query_cache_limit = 1M
query_cache_size        = 16M

log_error                = /var/log/mysql/error.log

slow_query_log  = 1
slow_query_log_file  = /var/log/mysql/slow.log
long_query_time = 0.1
expire_logs_days  = 10
max_binlog_size         = 100M

[mysqldump]
quick
quote-names
max_allowed_packet  = 16M

[isamchk]
key_buffer    = 16M

!includedir /etc/mysql/conf.d/

# cat /etc/mysql/conf.d/*

[mysqld]
innodb_file_per_table
innodb_rollback_on_timeout = ON
innodb_lock_wait_timeout = 10
# http://dev.mysql.com/doc/refman/5.5/en/innodb-auto-increment-handling.html#innodb-auto-increment-configurable
# This is needed to allow multiple LOAD DATA statements to execute parallely.
innodb_autoinc_lock_mode = 2
# To allow replication on lock mode 2.
binlog_format=row

That's pretty much entire error log there after server restart...

# > /var/log/mysql/error.log 
# /etc/init.d/mysql start
 * Starting MySQL (Percona Server) database server mysqld                [fail] 
# cat /var/log/mysql/error.log
130608 07:50:46 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
130608  7:50:46 [Warning] You need to use --log-bin to make --binlog-format work.
130608  7:50:46 [Note] Plugin 'FEDERATED' is disabled.
130608  7:50:46 InnoDB: The InnoDB memory heap is disabled
130608  7:50:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130608  7:50:46 InnoDB: Compressed tables use zlib 1.2.3
130608  7:50:46 InnoDB: Using Linux native AIO
130608  7:50:46 InnoDB: Initializing buffer pool, size = 128.0M
130608  7:50:46 InnoDB: Completed initialization of buffer pool
130608  7:50:46 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 668771298910
130608  7:50:46  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 668771504592
04:50:46 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 350507 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0x8430e83]
/usr/sbin/mysqld(handle_fatal_signal+0x4bc)[0x82eb6ac]
[0xb77c8500]
/usr/sbin/mysqld[0x8580080]
/usr/sbin/mysqld[0x85807c7]
/usr/sbin/mysqld[0x8565023]
/usr/sbin/mysqld[0x856685a]
/usr/sbin/mysqld[0x84f2353]
/usr/sbin/mysqld[0x85011d8]
/usr/sbin/mysqld[0x8501cd9]
/usr/sbin/mysqld[0x84f40c1]
/usr/sbin/mysqld[0x8549d00]
/usr/sbin/mysqld[0x850666e]
/usr/sbin/mysqld[0x84a230a]
/usr/sbin/mysqld[0x845c314]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x4e)[0x82eec5e]
/usr/sbin/mysqld[0x81d5888]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0xbab)[0x81da42b]
/usr/sbin/mysqld[0x81450ee]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x4fa)[0x81490aa]
/usr/sbin/mysqld(main+0x27)[0x813d327]
/lib/libc.so.6(__libc_start_main+0xe6)[0xb7469bc6]
/usr/sbin/mysqld[0x813d261]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
130608 07:50:46 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

# free -m
             total       used       free     shared    buffers     cached
Mem:          6144        412       5731          0          0        320
-/+ buffers/cache:         91       6052
Swap:         3072          0       3072

Valerii Kravchuk (valerii-kravchuk) on 2013-06-08

Changed in percona-server:
status:	Incomplete → New

Revision history for this message

Valerii Kravchuk (valerii-kravchuk) wrote on 2013-06-27:

#3

If you still have access to that datadir and had not solved the problem yet, please, send the output of:

ls -l /var/lib/mysql

and the entire error log (or at least part of it since last successful startup). Also check /var/log/messages for any suspicious messages at the moment of this problem.

Changed in percona-server:
status:	New → Incomplete

Revision history for this message

Artūras Šlajus (x11-arturaz) wrote on 2013-06-27:

#4

Download full text (18.7 KiB)

Nothing in syslog or messages.

root@spudder:/var/lib/mysql.crashed# ls -laR
.:
total 19578684
drwxr-xr-x 5 mysql mysql 4096 Jun 15 09:45 .
drwxr-xr-x 41 root root 4096 Jun 15 09:52 ..
-rw-r--r-- 1 mysql mysql 0 Jun 6 20:06 debian-5.5.flag
-rw-r----- 1 mysql mysql 5242880 Jun 15 09:45 ib_logfile0
-rw-r----- 1 mysql mysql 5242880 Jun 6 16:40 ib_logfile1
-rw-r----- 1 mysql mysql 20428357632 Jun 6 16:30 ibdata1
drwxr-xr-x 2 mysql mysql 4096 Jun 12 2012 mysql
-rw------- 1 mysql mysql 6 Jun 12 2012 mysql_upgrade_info
drwx------ 2 mysql mysql 4096 Jun 6 20:06 performance_schema
drwx------ 2 mysql mysql 4096 Aug 19 2012 spacegame_game

./mysql:
total 1052
drwxr-xr-x 2 mysql mysql 4096 Jun 12 2012 .
drwxr-xr-x 5 mysql mysql 4096 Jun 15 09:45 ..
-rw-rw---- 1 mysql mysql 0 Jun 12 2012 columns_priv.MYD
-rw-rw---- 1 mysql mysql 4096 Jun 12 2012 columns_priv.MYI
-rw-rw---- 1 mysql mysql 8820 Jun 12 2012 columns_priv.frm
-rw-rw---- 1 mysql mysql 440 Jun 12 2012 db.MYD
-rw-rw---- 1 mysql mysql 5120 Jun 12 2012 db.MYI
-rw-rw---- 1 mysql mysql 9582 Jun 12 2012 db.frm
-rw-rw---- 1 mysql mysql 0 Jun 12 2012 event.MYD
-rw-rw---- 1 mysql mysql 2048 Jun 12 2012 event.MYI
-rw-rw---- 1 mysql mysql 10223 Jun 12 2012 event.frm
-rw-rw---- 1 mysql mysql 0 Jun 12 2012 func.MYD
-rw-rw---- 1 mysql mysql 1024 Jun 12 2012 func.MYI
-rw-rw---- 1 mysql mysql 8665 Jun 12 2012 func.frm
-rw-r----- 1 mysql mysql 35 Jun 6 20:06 general_log.CSM
-rw-r----- 1 mysql mysql 0 Jun 12 2012 general_log.CSV
-rw-rw---- 1 mysql mysql 8776 Jun 12 2012 general_log.frm
-rw-r----- 1 mysql mysql 23240 Jun 6 20:06 help_category.MYD
-rw-r----- 1 mysql mysql 3072 Jun 6 20:06 help_category.MYI
-rw-r----- 1 mysql mysql 8700 Jun 12 2012 help_category.frm
-rw-r----- 1 mysql mysql 91999 Jun 6 20:06 help_keyword.MYD
-rw-r----- 1 mysql mysql 17408 Jun 6 20:06 help_keyword.MYI
-rw-r----- 1 mysql mysql 8612 Jun 12 2012 help_keyword.frm
-rw-r----- 1 mysql mysql 9432 Jun 6 20:06 help_relation.MYD
-rw-r----- 1 mysql mysql 19456 Jun 6 20:06 help_relation.MYI
-rw-r----- 1 mysql mysql 8630 Jun 12 2012 help_relation.frm
-rw-r----- 1 mysql mysql 470672 Jun 6 20:06 help_topic.MYD
-rw-r----- 1 mysql mysql 20480 Jun 6 20:06 help_topic.MYI
-rw-r----- 1 mysql mysql 8770 Jun 12 2012 help_topic.frm
-rw-rw---- 1 mysql mysql 0 Jun 12 2012 host.MYD
-rw-rw---- 1 mysql mysql 2048 Jun 12 2012 host.MYI
-rw-rw---- 1 mysql mysql 9510 Jun 12 2012 host.frm
-rw-r----- 1 mysql mysql 0 Jun 12 2012 ndb_binlog_index.MYD
-rw-r----- 1 mysql mysql 1024 Jun 12 2012 ndb_binlog_index.MYI
-rw-r----- 1 mysql mysql 8778 Jun 12 2012 ndb_binlog_index.frm
-rw-rw---- 1 mysql mysql 0 Jun 12 2012 plugin.MYD
-rw-rw---- 1 mysql mysql 1024 Jun 12 2012 plugin.MYI
-rw-rw---- 1 mysql mysql 8586 Jun 12 2012 plugin.frm
-rw-rw---- 1 mysql mysql 320 Jun 12 2012 proc.MYD
-rw-rw---- 1 mysql mysql 4096 Jun 12 2012 proc.MYI
-rw-rw---- 1 mysql mysql 9996 Jun 12 2012 proc.frm
-rw-rw---- 1 mysql mysql 0 Jun 12 2012 procs_priv.MYD
-rw-rw---- 1 mysql my...

Nothing in syslog or messages.

root@spudder:/var/lib/mysql.crashed# ls -laR
.:
total 19578684
drwxr-xr-x  5 mysql mysql        4096 Jun 15 09:45 .
drwxr-xr-x 41 root  root         4096 Jun 15 09:52 ..
-rw-r--r--  1 mysql mysql           0 Jun  6 20:06 debian-5.5.flag
-rw-r-----  1 mysql mysql     5242880 Jun 15 09:45 ib_logfile0
-rw-r-----  1 mysql mysql     5242880 Jun  6 16:40 ib_logfile1
-rw-r-----  1 mysql mysql 20428357632 Jun  6 16:30 ibdata1
drwxr-xr-x  2 mysql mysql        4096 Jun 12  2012 mysql
-rw-------  1 mysql mysql           6 Jun 12  2012 mysql_upgrade_info
drwx------  2 mysql mysql        4096 Jun  6 20:06 performance_schema
drwx------  2 mysql mysql        4096 Aug 19  2012 spacegame_game

./mysql:
total 1052
drwxr-xr-x 2 mysql mysql   4096 Jun 12  2012 .
drwxr-xr-x 5 mysql mysql   4096 Jun 15 09:45 ..
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 columns_priv.MYD
-rw-rw---- 1 mysql mysql   4096 Jun 12  2012 columns_priv.MYI
-rw-rw---- 1 mysql mysql   8820 Jun 12  2012 columns_priv.frm
-rw-rw---- 1 mysql mysql    440 Jun 12  2012 db.MYD
-rw-rw---- 1 mysql mysql   5120 Jun 12  2012 db.MYI
-rw-rw---- 1 mysql mysql   9582 Jun 12  2012 db.frm
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 event.MYD
-rw-rw---- 1 mysql mysql   2048 Jun 12  2012 event.MYI
-rw-rw---- 1 mysql mysql  10223 Jun 12  2012 event.frm
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 func.MYD
-rw-rw---- 1 mysql mysql   1024 Jun 12  2012 func.MYI
-rw-rw---- 1 mysql mysql   8665 Jun 12  2012 func.frm
-rw-r----- 1 mysql mysql     35 Jun  6 20:06 general_log.CSM
-rw-r----- 1 mysql mysql      0 Jun 12  2012 general_log.CSV
-rw-rw---- 1 mysql mysql   8776 Jun 12  2012 general_log.frm
-rw-r----- 1 mysql mysql  23240 Jun  6 20:06 help_category.MYD
-rw-r----- 1 mysql mysql   3072 Jun  6 20:06 help_category.MYI
-rw-r----- 1 mysql mysql   8700 Jun 12  2012 help_category.frm
-rw-r----- 1 mysql mysql  91999 Jun  6 20:06 help_keyword.MYD
-rw-r----- 1 mysql mysql  17408 Jun  6 20:06 help_keyword.MYI
-rw-r----- 1 mysql mysql   8612 Jun 12  2012 help_keyword.frm
-rw-r----- 1 mysql mysql   9432 Jun  6 20:06 help_relation.MYD
-rw-r----- 1 mysql mysql  19456 Jun  6 20:06 help_relation.MYI
-rw-r----- 1 mysql mysql   8630 Jun 12  2012 help_relation.frm
-rw-r----- 1 mysql mysql 470672 Jun  6 20:06 help_topic.MYD
-rw-r----- 1 mysql mysql  20480 Jun  6 20:06 help_topic.MYI
-rw-r----- 1 mysql mysql   8770 Jun 12  2012 help_topic.frm
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 host.MYD
-rw-rw---- 1 mysql mysql   2048 Jun 12  2012 host.MYI
-rw-rw---- 1 mysql mysql   9510 Jun 12  2012 host.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 ndb_binlog_index.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 ndb_binlog_index.MYI
-rw-r----- 1 mysql mysql   8778 Jun 12  2012 ndb_binlog_index.frm
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 plugin.MYD
-rw-rw---- 1 mysql mysql   1024 Jun 12  2012 plugin.MYI
-rw-rw---- 1 mysql mysql   8586 Jun 12  2012 plugin.frm
-rw-rw---- 1 mysql mysql    320 Jun 12  2012 proc.MYD
-rw-rw---- 1 mysql mysql   4096 Jun 12  2012 proc.MYI
-rw-rw---- 1 mysql mysql   9996 Jun 12  2012 proc.frm
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 procs_priv.MYD
-rw-rw---- 1 mysql mysql   4096 Jun 12  2012 procs_priv.MYI
-rw-rw---- 1 mysql mysql   8875 Jun 12  2012 procs_priv.frm
-rw-r----- 1 mysql mysql   1386 Jun 12  2012 proxies_priv.MYD
-rw-r----- 1 mysql mysql   5120 Jun 12  2012 proxies_priv.MYI
-rw-r----- 1 mysql mysql   8800 Jun 12  2012 proxies_priv.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 servers.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 servers.MYI
-rw-r----- 1 mysql mysql   8838 Jun 12  2012 servers.frm
-rw-rw---- 1 mysql mysql     35 Jun  6 20:06 slow_log.CSM
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 slow_log.CSV
-rw-rw---- 1 mysql mysql   8976 Jun 12  2012 slow_log.frm
-rw-rw---- 1 mysql mysql      0 Jun 12  2012 tables_priv.MYD
-rw-rw---- 1 mysql mysql   4096 Jun 12  2012 tables_priv.MYI
-rw-rw---- 1 mysql mysql   8955 Jun 12  2012 tables_priv.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 time_zone.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 time_zone.MYI
-rw-r----- 1 mysql mysql   8636 Jun 12  2012 time_zone.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 time_zone_leap_second.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 time_zone_leap_second.MYI
-rw-r----- 1 mysql mysql   8624 Jun 12  2012 time_zone_leap_second.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 time_zone_name.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 time_zone_name.MYI
-rw-r----- 1 mysql mysql   8606 Jun 12  2012 time_zone_name.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 time_zone_transition.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 time_zone_transition.MYI
-rw-r----- 1 mysql mysql   8686 Jun 12  2012 time_zone_transition.frm
-rw-r----- 1 mysql mysql      0 Jun 12  2012 time_zone_transition_type.MYD
-rw-r----- 1 mysql mysql   1024 Jun 12  2012 time_zone_transition_type.MYI
-rw-r----- 1 mysql mysql   8748 Jun 12  2012 time_zone_transition_type.frm
-rw-rw---- 1 mysql mysql    588 Jun 12  2012 user.MYD
-rw-rw---- 1 mysql mysql   2048 Jun 12  2012 user.MYI
-rw-rw---- 1 mysql mysql  10630 Jun 12  2012 user.frm

./performance_schema:
total 216
drwx------ 2 mysql mysql 4096 Jun  6 20:06 .
drwxr-xr-x 5 mysql mysql 4096 Jun 15 09:45 ..
-rw-rw---- 1 mysql mysql 8624 Jun  6 20:06 cond_instances.frm
-rw-rw---- 1 mysql mysql   61 Jun  6 20:06 db.opt
-rw-rw---- 1 mysql mysql 9220 Jun  6 20:06 events_waits_current.frm
-rw-rw---- 1 mysql mysql 9220 Jun  6 20:06 events_waits_history.frm
-rw-rw---- 1 mysql mysql 9220 Jun  6 20:06 events_waits_history_long.frm
-rw-rw---- 1 mysql mysql 8878 Jun  6 20:06 events_waits_summary_by_instance.frm
-rw-rw---- 1 mysql mysql 8854 Jun  6 20:06 events_waits_summary_by_thread_by_event_name.frm
-rw-rw---- 1 mysql mysql 8814 Jun  6 20:06 events_waits_summary_global_by_event_name.frm
-rw-rw---- 1 mysql mysql 8654 Jun  6 20:06 file_instances.frm
-rw-rw---- 1 mysql mysql 8800 Jun  6 20:06 file_summary_by_event_name.frm
-rw-rw---- 1 mysql mysql 8840 Jun  6 20:06 file_summary_by_instance.frm
-rw-rw---- 1 mysql mysql 8684 Jun  6 20:06 mutex_instances.frm
-rw-rw---- 1 mysql mysql 8776 Jun  6 20:06 performance_timers.frm
-rw-rw---- 1 mysql mysql 8758 Jun  6 20:06 rwlock_instances.frm
-rw-rw---- 1 mysql mysql 8605 Jun  6 20:06 setup_consumers.frm
-rw-rw---- 1 mysql mysql 8637 Jun  6 20:06 setup_instruments.frm
-rw-rw---- 1 mysql mysql 8650 Jun  6 20:06 setup_timers.frm
-rw-rw---- 1 mysql mysql 8650 Jun  6 20:06 threads.frm

./spacegame_game:
total 78564
drwx------ 2 mysql mysql     4096 Aug 19  2012 .
drwxr-xr-x 5 mysql mysql     4096 Jun 15 09:45 ..
-rw-r----- 1 mysql mysql     8758 Jun 12  2012 alliances.frm
-rw-r----- 1 mysql mysql     9450 Jun 12  2012 buildings.frm
-rw-r----- 1 mysql mysql    13328 Jun 12  2012 callbacks.frm
-rw-r----- 1 mysql mysql     8688 Jun 12  2012 chat_messages.frm
-rw-r----- 1 mysql mysql     8622 Jun 12  2012 combat_logs.frm
-rw-r----- 1 mysql mysql     8800 Jun 12  2012 construction_queue_entries.frm
-rw-rw---- 1 mysql mysql     8868 Jul  5  2012 cooldowns.frm
-rw-r----- 1 mysql mysql     9108 Jun 12  2012 cred_stats.frm
-rw-r----- 1 mysql mysql       65 Jun 12  2012 db.opt
-rw-r----- 1 mysql mysql     8648 Jun 12  2012 folliages.frm
-rw-rw---- 1 mysql mysql     8784 Jun 20  2012 fow_galaxy_entries.frm
-rw-rw---- 1 mysql mysql   180224 Jun  6 16:30 fow_galaxy_entries.ibd
-rw-r----- 1 mysql mysql     8842 Jun 12  2012 galaxies.frm
-rw-rw---- 1 mysql mysql     8806 Aug 10  2012 location_locks.frm
-rw-rw---- 1 mysql mysql   147456 Jun  6 16:30 location_locks.ibd
-rw-r----- 1 mysql mysql     9024 Jun 12  2012 market_offers.frm
-rw-rw---- 1 mysql mysql     8752 Aug 19  2012 market_rates.frm
-rw-rw---- 1 mysql mysql   114688 Jun  6 15:42 market_rates.ibd
-rw-r----- 1 mysql mysql     8764 Jun 12  2012 naps.frm
-rw-r----- 1 mysql mysql     8812 Jun 12  2012 notifications.frm
-rw-r----- 1 mysql mysql     8682 Jun 12  2012 objective_progresses.frm
-rw-r----- 1 mysql mysql     8850 Jun 12  2012 objectives.frm
-rw-r----- 1 mysql mysql     8600 Jun 12  2012 player_options.frm
-rw-rw---- 1 mysql mysql    10102 Jun 27  2012 players.frm
-rw-rw---- 1 mysql mysql 12582912 Jun  6 16:30 players.ibd
-rw-r----- 1 mysql mysql     8708 Jun 12  2012 quest_progresses.frm
-rw-r----- 1 mysql mysql     8732 Jun 12  2012 quests.frm
-rw-r----- 1 mysql mysql     8974 Jun 12  2012 route_hops.frm
-rw-r----- 1 mysql mysql     9548 Jun 12  2012 routes.frm
-rw-r----- 1 mysql mysql     8566 Jun 12  2012 schema_migrations.frm
-rw-r----- 1 mysql mysql     8800 Jun 12  2012 solar_systems.frm
-rw-rw---- 1 mysql mysql    10684 Jul  5  2012 ss_objects.frm
-rw-rw---- 1 mysql mysql 88080384 Jun  6 16:30 ss_objects.ibd
-rw-r----- 1 mysql mysql     8890 Jun 12  2012 technologies.frm
-rw-r----- 1 mysql mysql     8674 Jun 12  2012 tiles.frm
-rw-r----- 1 mysql mysql    13734 Jun 12  2012 units.frm
-rw-r----- 1 mysql mysql     8868 Jun 12  2012 wreckages.frm

root@spudder:/var/lib/mysql.crashed# /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql.crashed --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-error=error.log --pid-file=mysqld.pid --socket=mysqld.sock --port=3307 -v
130627 16:19:20 [Warning] option 'slow_query_log': boolean value '/var/log/mysql/slow.log' wasn't recognized. Set to OFF.
root@spudder:/var/lib/mysql.crashed# ls
debian-5.5.flag  error.log  ib_logfile0  ib_logfile1  ibdata1  mysql  mysql_upgrade_info  performance_schema  spacegame_game
root@spudder:/var/lib/mysql.crashed# ls -laR /usr/lib/mysql/plugin
/usr/lib/mysql/plugin:
total 4456
drwxr-xr-x 3 root root    4096 Jun  6 20:06 .
drwxr-xr-x 3 root root    4096 Jun 12  2012 ..
-rw-r--r-- 1 root root    5612 May 24 06:37 adt_null.so
-rw-r--r-- 1 root root    5692 May 24 06:37 auth.so
-rw-r--r-- 1 root root   13848 May 24 06:37 auth_pam.so
-rw-r--r-- 1 root root    9760 May 24 06:37 auth_pam_compat.so
-rw-r--r-- 1 root root    5544 May 24 06:37 auth_socket.so
-rw-r--r-- 1 root root    5672 May 24 06:37 auth_test_plugin.so
-rw-r--r-- 1 root root     227 May 24 06:00 daemon_example.ini
drwxr-xr-x 2 root root    4096 Jun  6 20:06 debug
-rw-r--r-- 1 root root 2802228 May 24 06:37 dialog.so
-rw-r--r-- 1 root root 1406468 May 24 06:30 handlersocket.a
-rw-r--r-- 1 root root     993 May 24 06:30 handlersocket.la
lrwxrwxrwx 1 root root      22 Jun  6 20:06 handlersocket.so -> handlersocket.so.0.0.0
lrwxrwxrwx 1 root root      22 Jun  6 20:06 handlersocket.so.0 -> handlersocket.so.0.0.0
-rw-r--r-- 1 root root  125784 May 24 06:37 handlersocket.so.0.0.0
-rw-r--r-- 1 root root    9704 May 24 06:37 libdaemon_example.so
-rw-r--r-- 1 root root    1770 May 24 06:37 libfnv1a_udf.a
-rw-r--r-- 1 root root     985 May 24 06:30 libfnv1a_udf.la
lrwxrwxrwx 1 root root      21 Jun  6 20:06 libfnv1a_udf.so -> libfnv1a_udf.so.0.0.0
lrwxrwxrwx 1 root root      21 Jun  6 20:06 libfnv1a_udf.so.0 -> libfnv1a_udf.so.0.0.0
-rw-r--r-- 1 root root    5336 May 24 06:37 libfnv1a_udf.so.0.0.0
-rw-r--r-- 1 root root    1836 May 24 06:37 libfnv_udf.a
-rw-r--r-- 1 root root     971 May 24 06:30 libfnv_udf.la
lrwxrwxrwx 1 root root      19 Jun  6 20:06 libfnv_udf.so -> libfnv_udf.so.0.0.0
lrwxrwxrwx 1 root root      19 Jun  6 20:06 libfnv_udf.so.0 -> libfnv_udf.so.0.0.0
-rw-r--r-- 1 root root    5332 May 24 06:37 libfnv_udf.so.0.0.0
-rw-r--r-- 1 root root    1770 May 24 06:37 libmurmur_udf.a
-rw-r--r-- 1 root root     992 May 24 06:30 libmurmur_udf.la
lrwxrwxrwx 1 root root      22 Jun  6 20:06 libmurmur_udf.so -> libmurmur_udf.so.0.0.0
lrwxrwxrwx 1 root root      22 Jun  6 20:06 libmurmur_udf.so.0 -> libmurmur_udf.so.0.0.0
-rw-r--r-- 1 root root    5336 May 24 06:37 libmurmur_udf.so.0.0.0
-rw-r--r-- 1 root root    5812 May 24 06:37 mypluglib.so
-rw-r--r-- 1 root root    5488 May 24 06:37 qa_auth_client.so
-rw-r--r-- 1 root root    5632 May 24 06:37 qa_auth_interface.so
-rw-r--r-- 1 root root    5548 May 24 06:37 qa_auth_server.so
-rw-r--r-- 1 root root   30740 May 24 06:37 semisync_master.so
-rw-r--r-- 1 root root    9836 May 24 06:37 semisync_slave.so

/usr/lib/mysql/plugin/debug:
total 3928
drwxr-xr-x 2 root root    4096 Jun  6 20:06 .
drwxr-xr-x 3 root root    4096 Jun  6 20:06 ..
-rw-r--r-- 1 root root   12756 May 24 06:12 adt_null.so
-rw-r--r-- 1 root root   21565 May 24 06:12 auth.so
-rw-r--r-- 1 root root   31636 May 24 06:12 auth_pam.so
-rw-r--r-- 1 root root   31170 May 24 06:12 auth_pam_compat.so
-rw-r--r-- 1 root root   11958 May 24 06:12 auth_socket.so
-rw-r--r-- 1 root root   20292 May 24 06:12 auth_test_plugin.so
-rw-r--r-- 1 root root 3531611 May 24 06:13 dialog.so
-rw-r--r-- 1 root root   28040 May 24 06:12 libdaemon_example.so
-rw-r--r-- 1 root root   14635 May 24 06:12 mypluglib.so
-rw-r--r-- 1 root root   17398 May 24 06:12 qa_auth_client.so
-rw-r--r-- 1 root root   20317 May 24 06:12 qa_auth_interface.so
-rw-r--r-- 1 root root   12591 May 24 06:12 qa_auth_server.so
-rw-r--r-- 1 root root  148771 May 24 06:12 semisync_master.so

root@spudder:/var/lib/mysql.crashed# cat error.log 
130627 16:18:14 [Warning] You need to use --log-bin to make --binlog-format work.
130627 16:18:14 [Note] Plugin 'FEDERATED' is disabled.
130627 16:18:14 InnoDB: The InnoDB memory heap is disabled
130627 16:18:14 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130627 16:18:14 InnoDB: Compressed tables use zlib 1.2.3
130627 16:18:14 InnoDB: Using Linux native AIO
130627 16:18:14 InnoDB: Initializing buffer pool, size = 128.0M
130627 16:18:14 InnoDB: Completed initialization of buffer pool
130627 16:18:14 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 668771298910
130627 16:18:14  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 668771504592
13:18:14 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 350507 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0x8430e83]
/usr/sbin/mysqld(handle_fatal_signal+0x4bc)[0x82eb6ac]
[0xb777c500]
/usr/sbin/mysqld[0x8580080]
/usr/sbin/mysqld[0x85807c7]
/usr/sbin/mysqld[0x8565023]
/usr/sbin/mysqld[0x856685a]
/usr/sbin/mysqld[0x84f2353]
/usr/sbin/mysqld[0x85011d8]
/usr/sbin/mysqld[0x8501cd9]
/usr/sbin/mysqld[0x84f40c1]
/usr/sbin/mysqld[0x8549d00]
/usr/sbin/mysqld[0x850666e]
/usr/sbin/mysqld[0x84a230a]
/usr/sbin/mysqld[0x845c314]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x4e)[0x82eec5e]
/usr/sbin/mysqld[0x81d5888]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0xbab)[0x81da42b]
/usr/sbin/mysqld[0x81450ee]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x4fa)[0x81490aa]
/usr/sbin/mysqld(main+0x27)[0x813d327]
/lib/libc.so.6(__libc_start_main+0xe6)[0xb741dbc6]
/usr/sbin/mysqld[0x813d261]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
130627 16:19:20 [Warning] You need to use --log-bin to make --binlog-format work.
130627 16:19:20 [Note] Plugin 'FEDERATED' is disabled.
130627 16:19:20 InnoDB: The InnoDB memory heap is disabled
130627 16:19:20 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130627 16:19:20 InnoDB: Compressed tables use zlib 1.2.3
130627 16:19:20 InnoDB: Using Linux native AIO
130627 16:19:20 InnoDB: Initializing buffer pool, size = 128.0M
130627 16:19:20 InnoDB: Completed initialization of buffer pool
130627 16:19:20 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 668771298910
130627 16:19:20  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 668771504592
13:19:20 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 350507 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0x8430e83]
/usr/sbin/mysqld(handle_fatal_signal+0x4bc)[0x82eb6ac]
[0xb7785500]
/usr/sbin/mysqld[0x8580080]
/usr/sbin/mysqld[0x85807c7]
/usr/sbin/mysqld[0x8565023]
/usr/sbin/mysqld[0x856685a]
/usr/sbin/mysqld[0x84f2353]
/usr/sbin/mysqld[0x85011d8]
/usr/sbin/mysqld[0x8501cd9]
/usr/sbin/mysqld[0x84f40c1]
/usr/sbin/mysqld[0x8549d00]
/usr/sbin/mysqld[0x850666e]
/usr/sbin/mysqld[0x84a230a]
/usr/sbin/mysqld[0x845c314]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x4e)[0x82eec5e]
/usr/sbin/mysqld[0x81d5888]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0xbab)[0x81da42b]
/usr/sbin/mysqld[0x81450ee]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x4fa)[0x81490aa]
/usr/sbin/mysqld(main+0x27)[0x813d327]
/lib/libc.so.6(__libc_start_main+0xe6)[0xb7426bc6]
/usr/sbin/mysqld[0x813d261]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.

Valerii Kravchuk (valerii-kravchuk) on 2013-06-27

Changed in percona-server:
status:	Incomplete → New

Revision history for this message

Valerii Kravchuk (valerii-kravchuk) wrote on 2014-09-16:

#5

Sorry for the delay with this bug report.

It may have something to do with http://bugs.mysql.com/bug.php?id=68025 and failed memory allocation. If you still have access to that datadir, please, try to start any version newer than 5.5.31 (5.5.32+0 on it and check for assertion failures or error messages.

Changed in percona-server:
status:	New → Incomplete

Revision history for this message

Shahriyar Rzayev (rzayev-sehriyar) wrote on 2018-01-25:

#6

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-2969

Percona Server moved to https://jira.percona.com/projects/PS

mysqld crashes on startup

Bug Description

Other bug subscribers

Remote bug watches