online upgrade GTID cause binlog damage in high write QPS situation
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Percona Server moved to https://jira.percona.com/projects/PS | Status tracked in 5.7 | |||||
5.5 |
Invalid
|
Undecided
|
Unassigned | |||
5.6 |
Triaged
|
High
|
Unassigned | |||
5.7 |
Invalid
|
Undecided
|
Unassigned |
Bug Description
percona verson:
$ rpm -qa |grep -i percona
Percona-
Percona-
Percona-
Percona-
Percona-
Percona-
Percona-
OS:
$ cat /etc/redhat-release
CentOS release 6.6 (Final)
$ uname -a
Linux 2.6.32-
Server:
aws c4.4xlarge with 3.4TB gp2 ssd volume
Runing 2 mysql instance in one server 3306(master)
Myql config file :
-------
[mysqld_multi]
mysqld = /usr/bin/
mysqladmin = /usr/bin/mysqladmin
[mysqld3306]
socket = /data/mysqldata
port = 3306
pid-file = /data/mysqldata
datadir = /data/mysqldata
user = mysql
server-id = 1523306
gtid_mode = ON
gtid_deployment
log_slave_updates = 1
enforce_
log_error = /data/mysqllog/
log_bin = /data/mysqllog/
relay_log = /data/mysqllog/
slow_query_
log_bin_
thread_handling = pool-of-threads
thread_
thread_pool_size = 22
slave-skip-
slave_compresse
log-slave-updates
skip-slave-start
innodb_
innodb_
innodb_
thread_cache_size = 32
innodb_io_capacity = 20000
innodb_
innodb_
innodb_
innodb_
innodb_
innodb_
innodb_
query_cache_type = OFF
long_query_time = 1
slow_query_log = ON
log-queries-
log-slow-
max_connect_errors = 200
max_connections = 10000
character_
character-
collation-server = utf8mb4_unicode_ci
init_connect='SET NAMES utf8mb4'
table_open_
open_files_
innodb_
innodb_
expire_logs_days = 7
-------
3307's conf is same with 3306 with diffrent server-id and port number
How to replay:
1、set up mysql cluster as 3306(master)
2、create table and user for test in 3306
create user test@127.0.0.1 identified by '123456';
grant all on test.* to test@127.0.0.1;
use test;
create table seq_test(
iid bigint(20) auto_increment,
dt1 timestamp,
dt2 timestamp default CURRENT_TIMESTAMP,
primary key(iid));
3、create a script to load data into mysql
$ cat /tmp/insert.py
#!/usr/bin/python
# -*- coding:utf-8 -*-
import MySQLdb
import datetime
db = MySQLdb.
cursor = db.cursor()
cnt = 0
while True:
cursor.
cnt = cnt + 1
if cnt % 10000 == 0:
print cnt
db.commit()
4、run 3 load data script instance to load data
python /tmp/insert.py &
python /tmp/insert.py &
python /tmp/insert.py &
now the wirte qps nearly 30000
5、turn off gtid_deployment
SET GLOBAL gtid_deployment
6、execute "show slave status\G;" in 3307 shows:
this because the binlog file in the master is damaged while turn GTID ON
I have tried 3 times ,the first time is ok,but the last 2 times are meet binlog damage problem.
tags: | added: gtid-deployment-step |
Thank you for the report.
> 5、turn off gtid_deployment _step in master (3306) _step = OFF;
> SET GLOBAL gtid_deployment
But according to https:/ /www.percona. com/doc/ percona- server/ 5.6/flexibility /online_ gtid_deployment .html gtid_deployment _step should be ON on slave only. This seems to not a bug for me.