wsrep_stt_extrabackup failed, if no [mysqld] in /etc/mysql/my.cnf

[Andrii Petrenko]

i'm running mysql with mysql_multi (http://www.sebastien-han.fr/blog/2012/08/06/multi-mysql-instances-with-galera/), so i have no [mysqld] section in my config.

i'm running mysql joiner for replication by command:

/usr/sbin/mysqld --bind-address=10.211.55.12 --socket=/var/run/mysql-002/mysqld.sock --port=3308 --pid-file=/var/run/mysql-002/mysqld.pid --datadir=/var/lib/mysql-002 --user=mysql --tmpdir=/var/lib/mysql-002/tmp --log-bin=/var/lib/mysql-002/mysql-bin --log-bin-index=/var/lib/mysql-002/binlog.index --log-error=/var/log/mysql-002/mysql.err --general_log_file=/var/log/mysql-002/mysql.log --general_log=1 --expire-logs-days=2 --wsrep_provider=/usr/lib/galera/libgalera_smm.so --wsrep_cluster_name=openstack --wsrep_cluster_address=gcomm://10.211.55.12:5675,10.211.55.13:5675?pc.wait_prim=no --wsrep_provider_options=gmcast.listen_addr=tcp://10.211.55.12:5675 --wsrep_slave_threads=12 --wsrep_sst_method=xtrabackup-v2 --wsrep_node_address=10.211.55.12 --wsrep_debug=on --binlog_format=ROW --default-storage-engine=innodb --max_connections=16384 --collation-server=utf8_general_ci --init-connect='SET NAMES utf8' --character-set-server=utf8 --myisam_sort_buffer_size=64M --wait_timeout=1800 --open_files_limit=102400 --table_open_cache=10000 --key_buffer_size=64M --max_allowed_packet=256M --query_cache_size=0 --query_cache_type=0 --sync_binlog=0 --binlog-ignore-db=zabbix --skip-external-locking --skip-name-resolve --innodb_file_format=Barracuda --innodb_file_per_table=1 --innodb_buffer_pool_size=3201M --innodb_log_file_size=640M --innodb_read_io_threads=8 --innodb_write_io_threads=8 --innodb_io_capacity=500 --innodb_flush_log_at_trx_commit=2 --innodb_flush_method=O_DIRECT --innodb_doublewrite=0 --innodb_autoinc_lock_mode=2 --innodb_locks_unsafe_for_binlog=1 --wsrep_sst_auth=wsrep_sst:dhkcfrLJ

in log i have an error:

Mar 19 09:33:40 ubuntu-12 -innobackupex-move: innobackupex version 2.3.4 based on MySQL server 5.6.24 Linux (x86_64)
(revision id: e80c779)
Mar 19 09:33:40 ubuntu-12 -innobackupex-move: Error: Move file ib_logfile0 to ./ib_logfile0 failed: Destination file
exists
Mar 19 09:33:40 ubuntu-12 -wsrep-sst-joiner: Move successful, removing /var/lib/mysql-002//.sst
Mar 19 09:33:40 ubuntu-12 -wsrep-sst-joiner: SST magic file /var/lib/mysql-002//xtrabackup_galera_info not found/read
able
Mar 19 09:33:40 ubuntu-12 -wsrep-sst-joiner: Cleanup after exit with status:2

when i add [mysqld] section replication complete successfully to the datadir directory in section [mysqld].

 percona-xtrabackup 2.3.4-1.trusty

logs and config attached.

[Sergei Glushchenko (sergei.glushchenko)]

In order for xtrabackup to work, SST script must pass --defaults-group=xxx option. I bet it doesn't.

[Andrii Petrenko (aplsms)]

Can you recommend better way to set multiple instances? in my feelings script is too old...

[Andrii Petrenko (aplsms)]

added to the command line: --defaults-group-suffix=2 --defaults-group=mysqld2

result is the same

can you check lines 650-660 of wsrep_sst_xtrabackup-v2:
depends of logging we have a different commands for INNMOVE: without syslog we are using --defaults-group=mysqld${WSREP_SST_OPT_CONF_SUFFIX} with syslog -- not using.

        INNOAPPLY="${INNOBACKUPEX_BIN} $disver $iapts --apply-log \$rebuildcmd \${DATA} 2>&1 | logger -p daemon.err -t ${ssystag}innobackupex-apply "
        INNOMOVE="${INNOBACKUPEX_BIN} --defaults-file=${WSREP_SST_OPT_CONF} $disver $impts --move-back --force-non-empty-directories \${DATA} 2>&1 | logger -p daemon.err -t ${ssystag}innobackupex-move "
        INNOBACKUP="${INNOBACKUPEX_BIN} --defaults-file=${WSREP_SST_OPT_CONF} $disver $iopts \$tmpopts \$INNOEXTRA --galera-info --stream=\$sfmt \$itmpdir 2> >(logger -p daemon.err -t ${ssystag}innobackupex-backup)"
    fi

else
    INNOAPPLY="${INNOBACKUPEX_BIN} $disver $iapts --apply-log \$rebuildcmd \${DATA} &>\${DATA}/innobackup.prepare.log"
    INNOMOVE="${INNOBACKUPEX_BIN} --defaults-file=${WSREP_SST_OPT_CONF} --defaults-group=mysqld${WSREP_SST_OPT_CONF_SUFFIX} $disver $impts --move-back --force-non-empty-directories \${DATA} &>\${DATA}/innobackup.move.log"
    INNOBACKUP="${INNOBACKUPEX_BIN} --defaults-file=${WSREP_SST_OPT_CONF} --defaults-group=mysqld${WSREP_SST_OPT_CONF_SUFFIX} $disver $iopts \$tmpopts \$INNOEXTRA --galera-info --stream=\$sfmt \$itmpdir 2>\${DATA}/innobackup.backup.log"
fi

[Kenn Takara (kenn-takara)]

Andrii is correct, the defaults-group suffix is not supplied when syslog is used. ouch, will look into this for the next release.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-532

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-825