pt-online-schema-change clobbers database table when interrupted

Bug #1421027 reported by monty solomon
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Percona Toolkit moved to https://jira.percona.com/projects/PT
Confirmed
High
Frank Cizmich

Bug Description

pt-online-schema-change clobbers the database table when interrupted.

After being interrupted, pt-osc renamed the new table to the original table even though nothing was inserted into it and then it deleted the old table.

pt-online-schema-change 2.2.13

CentOS release 6.5 (Final)
Linux 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Revision history for this message
monty solomon (monty+launchpad) wrote :

Here is a sample log excerpt from a failed aborted run

SHOW GLOBAL STATUS LIKE 'Threads_running'
SHOW GLOBAL STATUS LIKE 'Threads_running'
SELECT CONCAT(@@hostname, @@port)
SHOW VARIABLES
SHOW TABLES FROM `Clobbered_DB` LIKE 'template'
SHOW TRIGGERS FROM `Clobbered_DB` LIKE 'template'
/*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
USE `Clobbered_DB`
SHOW CREATE TABLE `Clobbered_DB`.`template`
/*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
EXPLAIN SELECT * FROM `Clobbered_DB`.`template` WHERE 1=1
SELECT table_schema, table_name FROM information_schema.key_column_usage WHERE referenced_table_schema='Clobbered_DB' AND referenced_table_name='template'
SHOW VARIABLES LIKE 'wsrep_on'
/*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
USE `Clobbered_DB`
SHOW CREATE TABLE `Clobbered_DB`.`template`
/*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
CREATE TABLE `Clobbered_DB`.`_template_new` (
ALTER TABLE `Clobbered_DB`.`_template_new` ADD COLUMN private tinyint(1) NOT NULL DEFAULT '0'
/*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
USE `Clobbered_DB`
SHOW CREATE TABLE `Clobbered_DB`.`_template_new`
/*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
CREATE TRIGGER `pt_osc_Clobbered_DB_template_del` AFTER DELETE ON `Clobbered_DB`.`template` FOR EACH ROW DELETE IGNORE FROM `Clobbered_DB`.`_template_new` /*excised*/
CREATE TRIGGER `pt_osc_Clobbered_DB_template_upd` AFTER UPDATE ON `Clobbered_DB`.`template` FOR EACH ROW REPLACE INTO `Clobbered_DB`.`_template_new` /*excised*/
CREATE TRIGGER `pt_osc_Clobbered_DB_template_ins` AFTER INSERT ON `Clobbered_DB`.`template` FOR EACH ROW REPLACE INTO `Clobbered_DB`.`_template_new` /*excised*/
EXPLAIN SELECT * FROM `Clobbered_DB`.`template` WHERE 1=1
SELECT /*!40001 SQL_NO_CACHE */ /*excised*/ LIMIT 1 /*first lower boundary*/
SELECT /*!40001 SQL_NO_CACHE */ /*excised*/ LIMIT 1 /*key_len*/
EXPLAIN SELECT /*!40001 SQL_NO_CACHE */ * FROM `Clobbered_DB`.`template` FORCE INDEX (`PRIMARY`) /*excised*/ /*key_len*/
EXPLAIN SELECT /*!40001 SQL_NO_CACHE */ /*excised*/, `id` FROM `Clobbered_DB`.`template` FORCE INDEX(`PRIMARY`) /*excised*/ LIMIT 999, 2 /*next chunk boundary*/
RENAME TABLE `Clobbered_DB`.`template` TO `Clobbered_DB`.`_template_old`, `Clobbered_DB`.`_template_new` TO `Clobbered_DB`.`template`
DROP TABLE IF EXISTS `Clobbered_DB`.`_template_old`
SHOW TABLES FROM `Clobbered_DB` LIKE '\_template\_new'

Revision history for this message
monty solomon (monty+launchpad) wrote :

This is a dangerous bug. It should be triaged with high priority and fixed as soon as possible.

Thanks.

Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

Hello monty,

I agree this should be high priority but I'm having trouble reproducing.
What type of interrupt caused it? A regular SIGINT (ctl+c) ?
Normally the tool is designed to handle these.
Can you provide the full command you used?
Also table schema and number of rows.
Does the interruption occur during row copying?
If you don't mind attaching the full PTDEBUG log, it would be useful.

Regards

Revision history for this message
monty solomon (monty+launchpad) wrote :

It appears that control-c handling was modified in the recent releases of percona-toolkit.

We had not seen the problem before the recent updates.

The users interrupted pt-online-schema-change using control-c.

It happened twice in the past two weeks. The first time the user realized the pt-online-schema-change command was pasted into the wrong window and aborted it using control-c.

The second time a different user was concerned about seeing detrimental performance impact due to running pt-online-schema-change and aborted using control-c. In that case pt-online-schema-change was being run by a script on a table in multiple databases, in order, and each control-c aborted one run. After aborting the pt-online-schema-change runs, three tables were clobbered (each in a different database).

Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

monty,

Still haven't been able to reproduce using various OSs including CentOS 6.5 (Final)
Can you consistently reproduce this?
If so, can you tell if the problem happens with version 2.2.12? (previous version)

Revision history for this message
monty solomon (monty+launchpad) wrote :

No, we can't consistently reproduce the problem and we have not seen the problem before this version.

Revision history for this message
monty solomon (monty+launchpad) wrote :
Download full text (3.6 KiB)

General log excerpt from another clobbered session

  38167216 Connect root@127.0.0.1 on db_16
  38167216 Query set autocommit=1
  38167216 Query SELECT @@SQL_MODE
  38167216 Query SHOW VARIABLES LIKE 'innodb\_lock_wait_timeout'
  38167216 Query SET SESSION innodb_lock_wait_timeout=1
  38167216 Query SHOW VARIABLES LIKE 'lock\_wait_timeout'
  38167216 Query SET SESSION lock_wait_timeout=60
  38167216 Query SHOW VARIABLES LIKE 'wait\_timeout'
  38167216 Query SET SESSION wait_timeout=10000
  38167216 Query SET @@SQL_QUOTE_SHOW_CREATE = 1/*!40101, @@SQL_MODE='NO_AUTO_VALUE_ON_ZERO,ONLY_FULL_GROUP_BY,NO_AUTO_VALUE_ON_ZERO,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'*/
  38167216 Query SELECT @@server_id /*!50038 , @@hostname*/
  38167216 Query SHOW VARIABLES LIKE 'wsrep_on'
  38167216 Query SHOW VARIABLES LIKE 'version%'
  38167216 Query SHOW ENGINES
  38167216 Query SHOW VARIABLES LIKE 'innodb_version'
  38167216 Query SELECT @@SERVER_ID
  38167216 Query SHOW GRANTS FOR CURRENT_USER()
  38167216 Query SHOW PROCESSLIST
  38167216 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
  38167216 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
  38167216 Query SELECT CONCAT(@@hostname, @@port)
  38167216 Query SHOW TABLES FROM `db_16` LIKE 'data'
  38167216 Query SHOW TRIGGERS FROM `db_16` LIKE 'data'
  38167216 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  38167216 Query USE `db_16`
  38167216 Query SHOW CREATE TABLE `db_16`.`data`
  38167216 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  38167216 Query EXPLAIN SELECT * FROM `db_16`.`data` WHERE 1=1
  38167216 Query SELECT table_schema, table_name FROM information_schema.key_column_usage WHERE referenced_table_schema='db_16' AND referenced_table_name='data'
  38167216 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  38167216 Query USE `db_16`
  38167216 Query SHOW CREATE TABLE `db_16`.`data`
  38167216 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  38167216 Query CREATE TABLE `db_16`.`_data_new` (
  38167216 Query ALTER TABLE `db_16`.`_data_new` ADD COLUMN flex_areas longtext
  38167216 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  38167216 Query USE `db_16`
  38167216 Query SHOW CREATE TABLE `db_16`.`_data_new`
  38167216 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  38167216 Query CREATE TRIGGER `pt_osc_db_16_data_del` AFTER DELETE ON `db_16`.`data`
  38167216 Query CREATE TRIGGER `pt_osc_db_16_data_upd` AFTER UPDATE ON `db_16`.`data`
  38167216 Query CREATE TRIGGER `pt_osc_db_16_data_ins` AFTER INSERT ON `db_16`.`data`
  38167216 Query EXPLAIN SELECT * FROM `db_16`.`data` WHERE 1=1
  38167216 Query SELECT /*!40001 SQL_NO_CACHE */ FROM `db_16`.`data` LIMIT 1 /*first lower boundary*/
  38167216 Query SELECT /*!40001 SQL_NO_CAC...

Read more...

Revision history for this message
monty solomon (monty+launchpad) wrote :
Download full text (3.6 KiB)

And another

  38167237 Connect root@127.0.0.1 on db_17
  38167237 Query set autocommit=1
  38167237 Query SELECT @@SQL_MODE
  38167237 Query SHOW VARIABLES LIKE 'innodb\_lock_wait_timeout'
  38167237 Query SET SESSION innodb_lock_wait_timeout=1
  38167237 Query SHOW VARIABLES LIKE 'lock\_wait_timeout'
  38167237 Query SET SESSION lock_wait_timeout=60
  38167237 Query SHOW VARIABLES LIKE 'wait\_timeout'
  38167237 Query SET SESSION wait_timeout=10000
  38167237 Query SET @@SQL_QUOTE_SHOW_CREATE = 1/*!40101, @@SQL_MODE='NO_AUTO_VALUE_ON_ZERO,ONLY_FULL_GROUP_BY,NO_AUTO_VALUE_ON_ZERO,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'*/
  38167237 Query SELECT @@server_id /*!50038 , @@hostname*/
  38167237 Query SHOW VARIABLES LIKE 'wsrep_on'
  38167237 Query SHOW VARIABLES LIKE 'version%'
  38167237 Query SHOW ENGINES
  38167237 Query SHOW VARIABLES LIKE 'innodb_version'
  38167237 Query SELECT @@SERVER_ID
  38167237 Query SHOW GRANTS FOR CURRENT_USER()
  38167237 Query SHOW PROCESSLIST
  38167237 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
  38167237 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
  38167237 Query SELECT CONCAT(@@hostname, @@port)
  38167237 Query SHOW TABLES FROM `db_17` LIKE 'data'
  38167237 Query SHOW TRIGGERS FROM `db_17` LIKE 'data'
  38167237 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  38167237 Query USE `db_17`
  38167237 Query SHOW CREATE TABLE `db_17`.`data`
  38167237 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  38167237 Query EXPLAIN SELECT * FROM `db_17`.`data` WHERE 1=1
  38167237 Query SELECT table_schema, table_name FROM information_schema.key_column_usage WHERE referenced_table_schema='db_17' AND referenced_table_name='data'
  38167237 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  38167237 Query USE `db_17`
  38167237 Query SHOW CREATE TABLE `db_17`.`data`
  38167237 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  38167237 Query CREATE TABLE `db_17`.`_data_new` (
  38167237 Query ALTER TABLE `db_17`.`_data_new` ADD COLUMN flex_areas longtext
  38167237 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  38167237 Query USE `db_17`
  38167237 Query SHOW CREATE TABLE `db_17`.`_data_new`
  38167237 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  38167237 Query CREATE TRIGGER `pt_osc_db_17_data_del` AFTER DELETE ON `db_17`.`data`
  38167237 Query CREATE TRIGGER `pt_osc_db_17_data_upd` AFTER UPDATE ON `db_17`.`data`
  38167237 Query CREATE TRIGGER `pt_osc_db_17_data_ins` AFTER INSERT ON `db_17`.`data`
  38167237 Query EXPLAIN SELECT * FROM `db_17`.`data` WHERE 1=1
  38167237 Query SELECT /*!40001 SQL_NO_CACHE */ FROM `db_17`.`data` LIMIT 1 /*first lower boundary*/
  38167237 Query SELECT /*!40001 SQL_NO_CACHE */ FROM `db_17`.`data` LIMIT 1 /*key_...

Read more...

Revision history for this message
monty solomon (monty+launchpad) wrote :
Download full text (3.6 KiB)

And another

  41678870 Connect root@127.0.0.1 on db_42
  41678870 Query set autocommit=1
  41678870 Query SELECT @@SQL_MODE
  41678870 Query SHOW VARIABLES LIKE 'innodb\_lock_wait_timeout'
  41678870 Query SET SESSION innodb_lock_wait_timeout=1
  41678870 Query SHOW VARIABLES LIKE 'lock\_wait_timeout'
  41678870 Query SET SESSION lock_wait_timeout=60
  41678870 Query SHOW VARIABLES LIKE 'wait\_timeout'
  41678870 Query SET SESSION wait_timeout=10000
  41678870 Query SET @@SQL_QUOTE_SHOW_CREATE = 1/*!40101, @@SQL_MODE='NO_AUTO_VALUE_ON_ZERO,ONLY_FULL_GROUP_BY,NO_AUTO_VALUE_ON_ZERO,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'*/
  41678870 Query SELECT @@server_id /*!50038 , @@hostname*/
  41678870 Query SHOW VARIABLES LIKE 'wsrep_on'
  41678870 Query SHOW VARIABLES LIKE 'version%'
  41678870 Query SHOW ENGINES
  41678870 Query SHOW VARIABLES LIKE 'innodb_version'
  41678870 Query SELECT @@SERVER_ID
  41678870 Query SHOW GRANTS FOR CURRENT_USER()
  41678870 Query SHOW PROCESSLIST
  41678870 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
  41678870 Query SHOW GLOBAL STATUS LIKE 'Threads_running'
  41678870 Query SELECT CONCAT(@@hostname, @@port)
  41678870 Query SHOW TABLES FROM `db_42` LIKE 'data'
  41678870 Query SHOW TRIGGERS FROM `db_42` LIKE 'data'
  41678870 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  41678870 Query USE `db_42`
  41678870 Query SHOW CREATE TABLE `db_42`.`data`
  41678870 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  41678870 Query EXPLAIN SELECT * FROM `db_42`.`data` WHERE 1=1
  41678870 Query SELECT table_schema, table_name FROM information_schema.key_column_usage WHERE referenced_table_schema='db_42' AND referenced_table_name='data'
  41678870 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  41678870 Query USE `db_42`
  41678870 Query SHOW CREATE TABLE `db_42`.`data`
  41678870 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  41678870 Query CREATE TABLE `db_42`.`_data_new` (
  41678870 Query ALTER TABLE `db_42`.`_data_new` ADD COLUMN flex_areas longtext
  41678870 Query /*!40101 SET @OLD_SQL_MODE := @@SQL_MODE, @@SQL_MODE := '', @OLD_QUOTE := @@SQL_QUOTE_SHOW_CREATE, @@SQL_QUOTE_SHOW_CREATE := 1 */
  41678870 Query USE `db_42`
  41678870 Query SHOW CREATE TABLE `db_42`.`_data_new`
  41678870 Query /*!40101 SET @@SQL_MODE := @OLD_SQL_MODE, @@SQL_QUOTE_SHOW_CREATE := @OLD_QUOTE */
  41678870 Query CREATE TRIGGER `pt_osc_db_42_data_del` AFTER DELETE ON `db_42`.`data`
  41678870 Query CREATE TRIGGER `pt_osc_db_42_data_upd` AFTER UPDATE ON `db_42`.`data`
  41678870 Query CREATE TRIGGER `pt_osc_db_42_data_ins` AFTER INSERT ON `db_42`.`data`
  41678870 Query EXPLAIN SELECT * FROM `db_42`.`data` WHERE 1=1
  41678870 Query SELECT /*!40001 SQL_NO_CACHE */ FROM `db_42`.`data` LIMIT 1 /*first lower boundary*/
  41678870 Query SELECT /*!40001 SQL_NO_CACHE */ FROM `db_42`.`data` LIMIT 1 /*key_...

Read more...

Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

If you are using version 2.2.13 , you can try removing line 10677 :

ReadKeyMini::ReadMode 0;

This was the only modification that was added to the signal handler. It doesn't affect my tests, but it's the only thing I can think of that might be causing problems.

Revision history for this message
monty solomon (monty+launchpad) wrote :

It appears that part of the code thought it was interrupted and part of the code didn't think it was interrupted because it renamed the tables but did not remove the triggers.

Revision history for this message
monty solomon (monty+launchpad) wrote :

It seems risky to continue to use version 2.2.13 in the case where it may need to be interrupted. I am rolling back to version 2.2.10 for now.

Revision history for this message
monty solomon (monty+launchpad) wrote :

Please check the code to see what causes the issue I mentioned in comment #11 to occur. That may help uncover the cause of the bug.

tags: added: pt-online-schema-change
Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

Hello monty

Did you try what I suggested in comment #10 ?
That would be the simplest solution while not producing regressions.

Otherwise we can try removing the ReadKeyMini module which *might* also be the cause of trouble.
I'll upload a patch removing the module but please try removoing the line I mentioned first, so we can rule that out.

Regards

Revision history for this message
monty solomon (monty+launchpad) wrote :

Trying to reproduce the problem I got the following error

2015-03-14T04:44:38 Error copying rows from `DB`.`template` to `DB`.`_template_new`: 2015-03-14T04:44:38 Use of uninitialized value $mode in numeric eq (==) at /usr/bin/pt-online-schema-change line 7845.

Revision history for this message
monty solomon (monty+launchpad) wrote :

I have not been able to intentionally reproduce the problem so I don't know how making the change outlined in comment #10 will fix anything.

Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

Hi monty,

Your comment #15 is evidence that a module introduced in 2.2.13 might be the culprit.

I've prepared a patch for 2.2.13 that removes that module while still keeping a few lines of code that fix the issue it was originally intended for. (FYI: https://bugs.launchpad.net/percona-toolkit/+bug/1396870)

If you could try this out and confirm it works well it would be great.
In any case we'll probably include this change in the next release since it seems a safer alternative.

Regards.

Changed in percona-toolkit:
status: New → Confirmed
Revision history for this message
monty solomon (monty+launchpad) wrote :

Was the change included in 2.2.14?

Revision history for this message
monty solomon (monty+launchpad) wrote :

By the way, I get a message stating that the current version is 2.2.13 even though 2.2.14 was released

# A software update is available:
# * The current version for Percona::Toolkit is 2.2.13.

# yum -q list percona-toolkit
Available Packages
percona-toolkit.noarch 2.2.14-1 percona-noarch

tags: added: i54636
tags: added: i57992
Revision history for this message
Chris (chris-church) wrote :

We are using 2.2.13 and just encountered this bug (in production!).

Was this fixed in 2.2.14? If not, is there a recommended workaround in place?

We encountered the same behavior where instead of rolling back, PTOSC basically truncated the original table.

Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

No workaround yet except using the next (unreleased version)
Attached here for convenience.

Changed in percona-toolkit:
importance: Undecided → High
assignee: nobody → Frank Cizmich (frank-cizmich)
Revision history for this message
Chris (chris-church) wrote :

Thanks, Frank.

When is the next version slated for official release?

Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

Chris,

If all goes well, next Wednesday, 26

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.