Debian package ignores GTID options when restarts server

Bug #1507812 reported by Sveta Smirnova on 2015-10-19
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Status tracked in 5.7
5.6
Fix Released
High
Tomislav Plavcic
5.7
Invalid
Undecided
Unassigned

Bug Description

When one upgrades Percona Server using Debian package, postinstall script restarts server few times. It can lead to situation when GTIDs sequence reset.

How to repeat.

1. Install Percona Server.
2. Start it with options:

binlog_gtid_simple_recovery = 1
enforce-gtid-consistency = 1
gtid_mode = ON

3. Insert some data, got GTIDs 1-N where N is any number, greater than 1.

4. Upgrade server

5. Insert some data, notice what GTIDs were reset to 1

Sveta Smirnova (svetasmirnova) wrote :

See also file #5 in the associated issue for detailed steps to reproduce.

Workaround: binlog_gtid_simple_recovery = 0

Sveta Smirnova (svetasmirnova) wrote :

Actually option binlog_gtid_simple_recovery = 1 strictly says what it can cause GTID resets if few binary logs without GTIDs were created after last log with GTIDs was closed, but this should not happen during upgrade.

Suggested fix: do not ignore GTID-related options, set in the configuration file.

Vlad Lesin (vlad-lesin) wrote :

There can be several variants of fix:

1) Do not ignore GTID related options on executing upgrade scripts. But all upgrade sql statements will be executed on slaves too, and this can be not desirable.

2) Force binlog_gtid_simple_recovery = 0 on first start after upgrade. This will work but can lead to lag during first start if there are big amount of binary log records.

3) Tell somehow to the server what exactly binary log file was the last just before the upgrade. In this case we could start usual procedure of searching gtid parameters with binlog_gtid_simple_recovery = 1 but use the passed binlog file as the last binary log for searching process.

Vlad, can you point me to the source code of script of Debian upgrade?

2) can be implemented by adding such start (and shutdown) to upgrade script itself. But it may take a long time to complete as you say.

Vlad Lesin (vlad-lesin) wrote :

The issue is upgrade script uses server's bootstap mode to execute some queries, and the server does not write GTID's in this mode. The simplified way to repeat:

1) Start it with options:

binlog_gtid_simple_recovery = 1
enforce-gtid-consistency = 1
gtid_mode = ON

2) Insert some data, got GTIDs 1-N where N is any number, greater than 1.

3) Stop the server and then execute any command in bootstrap mode. Example:

echo "insert into test.t1 (b) select b from test.t1;" | mysqld --bootstrap --user=mysql --skip-grant-tables

4) Start the server, GTIDs will begin with 1.

Vlad Lesin (vlad-lesin) wrote :

static int init_server_components()
{
...
  if (gtid_mode >= 1 && opt_bootstrap)
  {
    sql_print_warning("Bootstrap mode disables GTIDs. Bootstrap mode "
    "should only be used by mysql_install_db which initializes the MySQL "
    "data directory and creates system tables.");
    gtid_mode= 0;
  }
...
}

Vlad, check 5.7:

- The 5.6 change is cfc41cd167bc36c9bca390ab5084ddf6c11e93b8, "BUG#13992602: BINLOGGING STATEMENTS WITH GTID CRASHES IN BOOTSTRAP MODE"

- This was changed in 5.7: f2300ce39b97c66dc022a4396e00b9b27ebce878, "BUG#20980271: FIX GTID MODE TO WORK CORRECTLY IN --BOOTSTRAP/--INITIALIZE MODE"

The 5.7 patch is not merely removing the startup flag change, but more involved. At least 5.7 can be checked from user point of view whether upgrade in bootstrap mode is binlogged and not replicated OK, then a backport feasibility can be evaluated.

Vlad Lesin (vlad-lesin) wrote :

> At least 5.7 can be checked from user point of view whether upgrade in bootstrap mode is binlogged and not replicated OK, then a backport feasibility can be evaluated.

5.7 does not use bootstrap for upgrade, it uses --initialize-insecure mode instead.

20980271 suggests that GTIDs in 5.7 work in initialize mode the same as in bootstrap mode.

5.7?

Tomislav Plavcic (tplavcic) wrote :

5.7 shouldn't be affected as it doesn't run those scripts on upgrade, only on first install

OK, please update the bug status accordingly. Does it need a null merge to 5.7?

Tomislav Plavcic (tplavcic) wrote :

sorry updated the bug status, null merge was done before

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3317

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers