Comment 21 for bug 1098566

Ramesh Sivaraman (rameshvs02) wrote :

Server is crashing even with wsrep_sst_method=rsync with different error

dir info

datadir=/usr/local/mysql/data
basedir=/usr/local/mysql
innodb_data_home_dir=/dev/shm/inno_data
innodb_log_group_home_dir=/dev/shm/inno_data

Error info

2015-12-04 03:36:08 12101 [Note] InnoDB: Setting log file /dev/shm/inno_data/ib_logfile101 size to 48 MB
2015-12-04 03:36:08 12101 [ERROR] InnoDB: Cannot create /dev/shm/inno_data/ib_logfile1
2015-12-04 03:36:08 12101 [ERROR] Plugin 'InnoDB' init function returned error.
2015-12-04 03:36:08 12101 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2015-12-04 03:36:08 12101 [ERROR] Unknown/unsupported storage engine: InnoDB
2015-12-04 03:36:08 12101 [ERROR] Aborting

-rw-rw---- 1 mysql mysql 12582912 Dec 4 02:55 ibdata1
-rw-rw---- 1 mysql mysql 50331648 Dec 4 02:55 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 Dec 4 02:55 ib_logfile1
-rw-rw---- 1 mysql mysql 50331648 Dec 4 02:55 ib_logfile101

If we run mysql_install_db on joiner node, startup script starts mysqld with success message but in background mysqld process terminating with wrong SST state

[root@c-p130-u1400-240 mysql]# ./support-files/mysql.server start
 ERROR! MySQL (Percona XtraDB Cluster) is not running, but lock file (/var/lock/subsys/mysql) exists
Starting MySQL (Percona XtraDB Cluster)..State transfer in progress, setting sleep higher
. SUCCESS!
[root@c-p130-u1400-240 mysql]#

2015-12-04 03:41:30 12681 [Note] WSREP: SST received: 00000000-0000-0000-0000-000000000000:5
2015-12-04 03:41:30 12681 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.6.26-74.0-25.12-log' socket: '/usr/local/mysql/data/mysql.sock' port: 3306 Percona XtraDB Cluster binary (GPL) 5.6.26-25.12, Revision 624ef81, wsrep_25.12
2015-12-04 03:41:30 12681 [ERROR] WSREP: Application received wrong state:
        Received: 00000000-0000-0000-0000-000000000000
        Required: f3e24345-9a61-11e5-8823-eabc7692ebb8
2015-12-04 03:41:30 12681 [ERROR] WSREP: Application state transfer failed. This is unrecoverable condition, restart required.