mysqldump takes 5x as long with 5.7.20-19

Bug #1742751 reported by William Taylor on 2018-01-11
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Status tracked in 5.7
5.7
Invalid
Undecided
Unassigned

Bug Description

Originally posted this on the Percona forums with no response yet. This is probably a better place to post though.

After we upgraded to 5.7.20-19 from 5.7.19, mysql backups for this table jumped from 42 mins to 5hrs.
The backup time for the innodb table doubled but the tokudb table took more than 5 times longer.
I saw there was a memory leak fixed in mysqldump but unless there were other changes there I
don't see that being the issue.

Any ideas?

# Time: 2018-01-04T12:22:36.800014Z
# User@Host: root[root] @ localhost [] Id: 148138
# Schema: cdr Last_errno: 0 Killed: 0
# Query_time: 2568.367009 Lock_time: 0.000000 Rows_sent: 523136203 Rows_examined: 523136203 Rows_affected: 0
# Bytes_sent: 188647575431
SET timestamp=1515068556;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `cdr_main`;

# Time: 2018-01-04T12:22:56.091752Z
# User@Host: root[root] @ localhost [] Id: 148138
# Schema: cdr Last_errno: 0 Killed: 0
# Query_time: 19.015636 Lock_time: 0.000000 Rows_sent: 11859695 Rows_examined: 11859695 Rows_affected: 0
# Bytes_sent: 239650071
SET timestamp=1515068576;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `line_item_ratings`;

After upgrade to 5.7.20-19

# Time: 2018-01-10T20:18:38.510011Z
# User@Host: root[root] @ localhost [] Id: 29239
# Schema: cdr Last_errno: 1681 Killed: 0
# Query_time: 18269.061056 Lock_time: 0.000000 Rows_sent: 525785773 Rows_examined: 525785773 Rows_affected: 0
# Bytes_sent: 189596705933
SET timestamp=1515615518;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `cdr_main`;

# Time: 2018-01-10T20:19:24.966083Z
# User@Host: root[root] @ localhost [] Id: 29239
# Schema: cdr Last_errno: 1681 Killed: 0
# Query_time: 44.798296 Lock_time: 0.000000 Rows_sent: 11859695 Rows_examined: 11859695 Rows_affected: 0
# Bytes_sent: 239650071
SET timestamp=1515615564;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `line_item_ratings`;

SHOW TABLE STATUS:
Name: cdr_main
Engine: TokuDB
Version: 10
Row_format: tokudb_zlib
Rows: 526232788
Avg_row_length: 323
Data_length: 169998403381
Max_data_length: 9223372036854775807
Index_length: 35156877283
Data_free: 18446743920195218452
Auto_increment: 829412264
Create_time: 2016-07-11 11:16:33
Update_time: 2018-01-10 17:08:38
Check_time: NULL
Collation: latin1_swedish_ci
Checksum: NULL
Create_options:
Comment:

Name: line_item_ratings
Engine: InnoDB
Version: 10
Row_format: Dynamic
Rows: 6108827
Avg_row_length: 35
Data_length: 214663168
Max_data_length: 0
Index_length: 261586944
Data_free: 5242880
Auto_increment: NULL
Create_time: 2016-08-24 11:34:06
Update_time: NULL
Check_time: NULL
Collation: latin1_swedish_ci
Checksum: NULL
Create_options:
Comment:

tags: added: performance regression
Roel Van de Paar (roel11) wrote :

@Shako, can you please test this as discussed? Thank you

Roel Van de Paar (roel11) wrote :

Discussion in the thread.

Roel Van de Paar (roel11) wrote :

Tests do not show the issue. If we receive further OS/settings info, we can test again.

William Taylor (williamt-sonic) wrote :

It looks like its not the new mysql version.
It appears to be this kernel "kernel-3.10.0-693.11.6.el7.x86_64"
rolling back to "kernel-3.10.0-693.11.1.el7.x86_64" seems to have resolved the problems.
This is on CentOS Linux release 7.4.1708 (Core)

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3772

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers