Comment 18 for bug 1516919

With deployment step on, the master does binlog correctly. All my tests show this works fine.

--
Jeremy Tinley
<email address hidden>

> On Mar 28, 2017, at 8:38 PM, Laurynas Biveinis <email address hidden> wrote:
>
> The startup variable setting issue needs to be fixed too, but the one I
> was talking about before is master with gtid_deployment_step=ON not
> binlogging its workload, which seems to be related to your latter
> comment.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1516919
>
> Title:
> online upgrade GTID cause binlog damage in high write QPS situation
>
> Status in Percona Server:
> Invalid
> Status in Percona Server 5.5 series:
> Invalid
> Status in Percona Server 5.6 series:
> Triaged
> Status in Percona Server 5.7 series:
> Invalid
>
> Bug description:
> percona verson:
> $ rpm -qa |grep -i percona
> Percona-Server-client-56-5.6.27-rel75.0.el6.x86_64
> Percona-Server-tokudb-56-5.6.27-rel75.0.el6.x86_64
> Percona-Server-devel-56-5.6.27-rel75.0.el6.x86_64
> Percona-Server-shared-56-5.6.27-rel75.0.el6.x86_64
> Percona-Server-server-56-5.6.27-rel75.0.el6.x86_64
> Percona-Server-test-56-5.6.27-rel75.0.el6.x86_64
> Percona-Server-56-debuginfo-5.6.27-rel75.0.el6.x86_64
>
> OS:
> $ cat /etc/redhat-release
> CentOS release 6.6 (Final)
> $ uname -a
> Linux 2.6.32-504.16.2.el6.x86_64 #1 SMP Wed Apr 22 06:48:29 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
>
>
> Server:
> aws c4.4xlarge with 3.4TB gp2 ssd volume
>
>
> Runing 2 mysql instance in one server 3306(master)<-3307(slave)
>
> Myql config file :
> ------------------------------------------------------------------------------------------------------------------------------------------
> [mysqld_multi]
> mysqld = /usr/bin/mysqld_safe
> mysqladmin = /usr/bin/mysqladmin
>
> [mysqld3306]
> socket = /data/mysqldata/mysql3306/mysql3306.sock
> port = 3306
> pid-file = /data/mysqldata/mysql3306/bjzw3306.pid
> datadir = /data/mysqldata/mysql3306
> user = mysql
> server-id = 1523306
>
>
> gtid_mode = ON
> gtid_deployment_step = ON
> log_slave_updates = 1
> enforce_gtid_consistency = 1
>
>
> log_error = /data/mysqllog/mysql3306/yxmysql-error.log
> log_bin = /data/mysqllog/mysql3306/yxmysql-bin.log
> relay_log = /data/mysqllog/mysql3306/yxmysql-relay-bin.log
> slow_query_log_file=/data/mysqllog/mysql3306/slow-queries.log
>
> log_bin_trust_function_creators = 1
>
> thread_handling = pool-of-threads
> thread_pool_max_threads = 100000
> thread_pool_size = 22
>
> slave-skip-errors=1062
> slave_compressed_protocol=ON
>
> log-slave-updates
> skip-slave-start
>
> innodb_file_per_table = 1
> innodb_file_format=BARRACUDA
>
> innodb_thread_concurrency = 0
> thread_cache_size = 32
>
> innodb_io_capacity = 20000
> innodb_read_io_threads = 8
> innodb_write_io_threads = 8
>
> innodb_log_file_size = 56M
> innodb_log_files_in_group = 3
> innodb_log_buffer_size = 8M
>
> innodb_buffer_pool_size = 56M
> innodb_buffer_pool_instances = 2
>
> query_cache_type = OFF
>
> long_query_time = 1
> slow_query_log = ON
> log-queries-not-using-indexes
> log-slow-admin-statements
> max_connect_errors = 200
> max_connections = 10000
> character_set_server = utf8mb4
> character-set-client-handshake = FALSE
> collation-server = utf8mb4_unicode_ci
> init_connect='SET NAMES utf8mb4'
> table_open_cache=10240
> open_files_limit=20480
> innodb_flush_log_at_trx_commit=0
> innodb_open_files=10000
> expire_logs_days = 7
>
> --------------------------------------------------END-------------------------------------------------------------------------------------
> 3307's conf is same with 3306 with diffrent server-id and port number
>
>
> How to replay:
> 1、set up mysql cluster as 3306(master)<--3307(slave)
> 2、create table and user for test in 3306
> create user test@127.0.0.1 identified by '123456';
> grant all on test.* to test@127.0.0.1;
> use test;
> create table seq_test(
> iid bigint(20) auto_increment,
> dt1 timestamp,
> dt2 timestamp default CURRENT_TIMESTAMP,
> primary key(iid));
>
> 3、create a script to load data into mysql
> $ cat /tmp/insert.py
> #!/usr/bin/python
> # -*- coding:utf-8 -*-
>
> import MySQLdb
> import datetime
>
> db = MySQLdb.connect('127.0.0.1','test','123456', '', 3306)
> cursor = db.cursor()
> cnt = 0
> while True:
> cursor.execute('''insert into test.seq_test(dt1) values('%s');''' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))
> cnt = cnt + 1
> if cnt % 10000 == 0:
> print cnt
> db.commit()
>
> 4、run 3 load data script instance to load data
> python /tmp/insert.py &
> python /tmp/insert.py &
> python /tmp/insert.py &
>
> now the wirte qps nearly 30000
>
> 5、turn off gtid_deployment_step in master (3306)
> SET GLOBAL gtid_deployment_step = OFF;
>
> 6、execute "show slave status\G;" in 3307 shows:
> Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the first event 'yxmysql-bin.000001' at 151, the last event read from '/data/mysqllog/mysql3306/yxmysql-bin.000001' at 464889689, the last byte read from '/data/mysqllog/mysql3306/yxmysql-bin.000001' at 464889708.'
>
> this because the binlog file in the master is damaged while turn GTID
> ON
>
> I have tried 3 times ,the first time is ok,but the last 2 times are
> meet binlog damage problem.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/percona-server/+bug/1516919/+subscriptions