innobackupex should not allow --no-lock --galera-info

[Jay Janssen]

5.5.27.

If, on a DONOR node, I add --no_lock to /usr/bin/wsrep_sst_xtrabackup INNOBACKUPEX_ARGS like this:

INNOBACKUPEX_ARGS="--no-lock"

Then SST fails on the JOINER node with this error:

120908 17:31:44 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup 'joiner' '192.168.70.4' '' '/var/lib/mysql/' '/etc/my.cnf' '27134' 2>sst.err: 32 (Broken pipe)
120908 17:31:44 [ERROR] WSREP: Failed to parse uuid:seqno pair: 'xtrabackup process ended without creating '/var/lib/mysql//xtrabackup_galera_info''
120908 17:31:44 [ERROR] WSREP: SST failed: 22 (Invalid argument)
120908 17:31:44 [ERROR] Aborting

There is no content in the sst.err on the JOINER.

I know adding this argument worked correctly in 5.5.24, but seems broken in 5.5.27

[Raghavendra D Prabhu (raghavendra-prabhu)]

Confirmed.

Now, this fails because,

from innobackupex:
===========
    if (!$option_no_lock) {
        # make a prep copy before locking tables, if using rsync
        backup_files(1);

        # flush tables with read lock
        mysql_lockall();
    }
=======

and write_galera_info is called from mysql_lockall

write_galera_info mainly does:

    # get binlog position
    mysql_send "SHOW STATUS LIKE 'wsrep_local_state_uuid';";
    mysql_send "SHOW STATUS LIKE 'wsrep_last_committed';";

Now, it needs to checked if above does require FTWRL or not.

[Vadim Tkachenko (vadim-tk)]

we need to have FTWRL to get consistent Galera position.
--no-lock can not be used for SST.

Later when Galera supports lockless snapshot we can handle it differently.

I am setting bug as "Invalid"

[Raghavendra D Prabhu (raghavendra-prabhu)]

During a SST from the donor node, writes are disabled, so it is interesting to see if wsrep_local_state_uuid and wsrep_last_committed change during SST or not (my guess is they do not).

Also, adding Xtrabackup project here since changes, if any, are required in that.

[Raghavendra D Prabhu (raghavendra-prabhu)]

@Vadim, Thanks, so locking is required then.

However, if both --no-lock and --galera-info are given, innobackupex code should fail or atleast print a warning to the error log rather than silently fail with broken pipe as in case of PXC.

[Raghavendra D Prabhu (raghavendra-prabhu)]

Currently, there is nothing in donor innobackup.backup.log about this and joiner error log only has:

120927 22:20:25 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup 'joiner' '10.0.2.154' '' '/var/lib/mysql/' '/etc/my.cnf' '3629' 2>sst.err: 32 (Broken pipe)
120927 22:20:25 [ERROR] WSREP: Failed to parse uuid:seqno pair: 'xtrabackup process ended without creating '/var/lib/mysql//xtrabackup_galera_info''

[Raghavendra D Prabhu (raghavendra-prabhu)]

Actually, Galera, atleast now, supports wsrep replication position in InnoDB headers, much like how the replication position is stored and recovered during prepare phase of Xtrabackup.

According to discussion I had, it stored as follows:

""
Wsrep GTID is found from InnoDB sys_header + TRX_SYS_WSREP_XID_INFO, see:
* storage/innobase/include/trx0sys.h for TRX_SYS_WSREP_XID_INFO and other definitions
* storage/innobase/trx/trx0sys.c:trx_sys_read_wsrep_checkpoint() for reading XID from sys_header
* sql/wsrep_mysqld.h and sql/wsrep_utils.cc how to read GTID from XID (wsrep_xid_{uuid|seqno})
* XID is defined in sql/handler.h and storage/innobase/include/trx0xa.h
""

May be a different bug can be opened for this if required.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-83