Feature req: innodb_log_file_size smarter on startup

[Arjen Lentz]

Feature request:
On InnoDB startup, check whether the iblog files exist, and what size they are.
Feed that size back into the innodb_log_file_size, regardless of what it was.
If it was different, you can write a warning to errorlog, but no error/abort.

This little tweak would allow us to increase the default log file size, without breaking existing installations.

[Arjen Lentz (arjen-lentz)]

OurDelta would like to see this for 5.1
Backport to 5.0 ok but not urgent.

[Arjen Lentz (arjen-lentz)]

=== modified file 'storage/xtradb/handler/ha_innodb.cc'
--- storage/xtradb/handler/ha_innodb.cc 2009-09-09 21:06:57 +0000
+++ storage/xtradb/handler/ha_innodb.cc 2009-10-14 03:12:52 +0000
@@ -2405,6 +2405,9 @@
  /* Get the current high water mark format. */
  innobase_file_format_check = (char*) trx_sys_file_format_max_get();

+ /* Get the current log file size. */
+ innobase_log_file_size = srv_log_file_size;
+
  DBUG_RETURN(FALSE);
 error:
  DBUG_RETURN(TRUE);

=== modified file 'storage/xtradb/srv/srv0start.c'
--- storage/xtradb/srv/srv0start.c 2009-09-16 17:43:01 +0000
+++ storage/xtradb/srv/srv0start.c 2009-10-14 03:17:20 +0000
@@ -547,6 +547,21 @@
 }

 /*************************************************************************
+Calculates the number of database pages which are represented by the
+file size in bytes. */
+static
+ulint
+srv_calc_pages(
+/*============*/
+ /* out: file size in database pages */
+ ulint size, /* in: low 32 bits of file size in bytes */
+ ulint size_high) /* in: high 32 bits of file size in bytes */
+{
+ return((size >> UNIV_PAGE_SIZE_SHIFT)
+ | (size_high << (32 - UNIV_PAGE_SIZE_SHIFT));
+}
+
+/*************************************************************************
 Creates or opens the log files and closes them. */
 static
 ulint
@@ -611,8 +626,9 @@
   ret = os_file_get_size(files[i], &size, &size_high);
   ut_a(ret);

- if (size != srv_calc_low32(srv_log_file_size)
- || size_high != srv_calc_high32(srv_log_file_size)) {
+ if ((size != srv_calc_low32(srv_log_file_size)
+ || size_high != srv_calc_high32(srv_log_file_size))
+ && log_file_has_been_opened) {

    fprintf(stderr,
     "InnoDB: Error: log file %s is"
@@ -625,6 +641,7 @@

    return(DB_ERROR);
   }
+ srv_log_file_size = srv_calc_pages(size, size_high);
  } else {
   *log_file_created = TRUE;

[Arjen Lentz (arjen-lentz)]

(sorry, mistake in the feedback to the system variables, fixed patch below)

In a nutshell, this patch allows the default log file size to be changed (in the compiled-in default, or my.cnf) for new installations, without breaking existing installations.

It does the following: if the iblogfiles already exist, and size differs from innodb_log_file_size, issue a warning but not abort; it'll also update the system variable with the actual log file size.

For an existing installation, the user can simply update the my.cnf with the actual file size to get rid of the warning, or do the procedure for changing the log file size.

=== modified file 'storage/xtradb/handler/ha_innodb.cc'
--- storage/xtradb/handler/ha_innodb.cc 2009-09-09 21:06:57 +0000
+++ storage/xtradb/handler/ha_innodb.cc 2009-10-14 03:40:05 +0000
@@ -2405,6 +2405,10 @@
  /* Get the current high water mark format. */
  innobase_file_format_check = (char*) trx_sys_file_format_max_get();

+ /* Get the current log file size. */
+ innobase_log_file_size = srv_log_file_size;
+ innobase_log_file_size *= UNIV_PAGE_SIZE;
+
  DBUG_RETURN(FALSE);
 error:
  DBUG_RETURN(TRUE);

=== modified file 'storage/xtradb/srv/srv0start.c'
--- storage/xtradb/srv/srv0start.c 2009-09-16 17:43:01 +0000
+++ storage/xtradb/srv/srv0start.c 2009-10-14 03:26:54 +0000
@@ -547,6 +547,21 @@
 }

 /*************************************************************************
+Calculates the number of database pages which are represented by the
+file size in bytes. */
+static
+ulint
+srv_calc_pages(
+/*============*/
+ /* out: file size in database pages */
+ ulint size, /* in: low 32 bits of file size in bytes */
+ ulint size_high) /* in: high 32 bits of file size in bytes */
+{
+ return((size >> UNIV_PAGE_SIZE_SHIFT)
+ | (size_high << (32 - UNIV_PAGE_SIZE_SHIFT));
+}
+
+/*************************************************************************
 Creates or opens the log files and closes them. */
 static
 ulint
@@ -623,7 +638,12 @@
     (ulong) srv_calc_high32(srv_log_file_size),
     (ulong) srv_calc_low32(srv_log_file_size));

- return(DB_ERROR);
+ if (log_file_has_been_opened) {
+
+ return(DB_ERROR);
+ }
+
+ srv_log_file_size = srv_calc_pages(size, size_high);
   }
  } else {
   *log_file_created = TRUE;

[Arjen Lentz (arjen-lentz)]

tnx to Antony Curtis for this.

[Vadim Tkachenko (vadim-tk)]

Can InnoDB actually increase log_files automatically (i.e. from default 5M to 50M ) ?
In current way it will cause a lot of confusing, you set 1000M, but system still runs on 5M

[Arjen Lentz (arjen-lentz)]

Hi Vadim

On 14/10/2009, at 2:54 PM, Vadim Tkachenko wrote:
> Can InnoDB actually increase log_files automatically (i.e. from
> default 5M to 50M ) ?

That'd be nice, if possible.
But also in some cases you'd want to reduce it, if someone previously
set up something much too big.

> In current way it will cause a lot of confusing, you set 1000M, but
> system still runs on 5M

It'll give the warning in errorlog/syslog, which is better than the
current behaviour of aborting - that hinders progress even for new
installations.

If you can do a resize up and down, that'd be excellent!

Cheers,
Arjen.
--
Arjen Lentz, Exec.Director @ Open Query (http://openquery.com)
Exceptional Services for MySQL at a fixed budget.

Follow our blog at http://openquery.com/blog/
OurDelta: enhanced builds for MySQL @ http://ourdelta.org

[Vadim Tkachenko (vadim-tk)]

Yasufumi is going to implement auto-increasing of log_size if value in my.cnf (or in defaults) is bigger than current log file.

[Stewart Smith (stewart)]

All development of XtraDB has moved under the Percona Server project - https://launchpad.net/percona-server - If this bug can be reproduced against current Percona Server, please file this bug against percona-server (you can simply do so by using the "Also affects project" link above).

Thanks,
Stewart Smith
Director of Server Development
Percona.

[Laurynas Biveinis (laurynas-biveinis)]

Fixed by 5.6 online log resizing

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-2515