pt-table-checksum fails to find pxc nodes when wsrep_node_incoming_address is set to AUTO

Bug #1399789 reported by Marc Castrovinci on 2014-12-05
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Toolkit moved to https://jira.percona.com/projects/PT
Fix Released
High
Frank Cizmich

Bug Description

Per this fix https://bugs.launchpad.net/percona-toolkit/+bug/1217466 , wsrep_node_incoming_address is now being used as a way to uniquely identify a cluster node.

However, our cluster sets this value to AUTO, so all the nodes are being seen as non-unique causing no nodes to be found. If I downgrade to version 2.2.11, the error goes away because server-id is being used again.

A possible fix could be to check the server-id first, and if thats not unique then check wsrep_node_incoming_address ( or visa versa )

percona-toolkit version: 2.2.12
 Server version: 5.6.20-68.0-56-log Percona XtraDB Cluster (GPL), Release 25.7, wsrep_25.7.r4126

Reproduce:

root@CCCCCCCCC:~# pt-table-checksum --version
pt-table-checksum 2.2.12
root@CCCCCCCCCC:~# pt-table-checksum --recursion-method=cluster u=XXXXXX,p=XXXXXXX= --empty-replicate-table
Diffs cannot be detected because no cluster nodes were found. Please read the --recursion-method documentation for information.
CCCCCCCCCCCCCCC is a cluster node but no other nodes or regular replicas were found. Use --recursion-method=dsn to specify the other nodes in the cluster.
root@CCCCCCCCCCC:~# mysql -e "show variables like 'wsrep_node_incoming_address';"
+-----------------------------+-------+
| Variable_name | Value |
+-----------------------------+-------+
| wsrep_node_incoming_address | AUTO |
+-----------------------------+-------+

Downgraded:

root@AAAAAAAA:/tmp# pt-table-checksum --recursion-method=cluster u=XXXXXXX,p=XXXXXXX --empty-replicate-table
Not checking replica lag on BBBBBBBBBBBBBBBBBBBBBBBBB because it is a cluster node.
Not checking replica lag on AAAAAAAAAAAAAAAAAAAAAAAA because it is a cluster node.
            TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE
12-05T20:37:54 0 0 0 1 0 0.047 mysql.columns_priv
12-05T20:37:54 0 0 2 1 0 0.067 mysql.db
12-05T20:37:54 0 0 0 1 0 0.086 mysql.event
12-05T20:37:54 0 0 0 1 0 0.043 mysql.func
12-05T20:37:55 0 0 40 1 0 0.076 mysql.help_category
12-05T20:37:55 0 0 485 1 0 0.099 mysql.help_keyword
12-05T20:37:55 0 0 1090 1 0 0.045 mysql.help_relation
12-05T20:37:55 0 0 533 1 0 0.044 mysql.help_topic
12-05T20:37:55 0 0 0 1 0 0.036 mysql.ndb_binlog_index
12-05T20:37:55 0 0 0 1 0 0.047 mysql.plugin
12-05T20:37:55 0 0 0 1 0 0.042 mysql.proc
12-05T20:37:55 0 0 0 1 0 0.046 mysql.procs_priv
12-05T20:37:55 0 0 2 1 0 0.041 mysql.proxies_priv
12-05T20:37:55 0 0 0 1 0 0.051 mysql.servers
12-05T20:37:55 0 0 0 1 0 0.039 mysql.tables_priv
12-05T20:37:55 0 0 0 1 0 0.047 mysql.time_zone
12-05T20:37:55 0 0 0 1 0 0.051 mysql.time_zone_leap_second
12-05T20:37:56 0 0 0 1 0 0.893 mysql.time_zone_name
12-05T20:37:56 0 0 0 1 0 0.050 mysql.time_zone_transition
12-05T20:37:56 0 0 0 1 0 0.037 mysql.time_zone_transition_type
12-05T20:37:56 0 0 7 1 0 0.044 mysql.user
root@AAAAAAAAAAA:/tmp# mysql -e "show variables like 'wsrep_node_incoming_address';"
+-----------------------------+-------+
| Variable_name | Value |
+-----------------------------+-------+
| wsrep_node_incoming_address | AUTO |
+-----------------------------+-------+

Related branches

Changed in percona-toolkit:
status: New → In Progress
milestone: none → 2.2.13
importance: Undecided → High
assignee: nobody → Frank Cizmich (frank-cizmich)
Frank Cizmich (frank-cizmich) wrote :

Attached is a version that uses a combination of variables to obtain a unique id.

tags: added: pt-table-checksum pxc
Kenny Gryp (gryp) wrote :

Arunjith had it too with a customer. I made a very quick patch that solved it for that case, changing to wsrep_node_address instead of wsrep_node_incoming_address

Changed in percona-toolkit:
status: In Progress → Fix Committed
Changed in percona-toolkit:
status: Fix Committed → Fix Released

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PT-384

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers