disk space grows by >500% while preparing a backup

Bug #950334 reported by Daniel Frett
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Percona XtraBackup moved to https://jira.percona.com/projects/PXB
Fix Released
High
Alexey Kopytov
2.0
Fix Released
High
Alexey Kopytov
2.1
Fix Released
High
Alexey Kopytov

Bug Description

when preparing a backup of a database with a large amount (100,000+) of small tables using innodb_file_per_table, each table grows to at least 1MB in size. This can result in a large increase in disk usage.

The problem appears to be caused by a call to _fil_io in fil0fil.c to load block_offset 0 of a tablespace that hasn't been loaded yet. Before the tablespace is loaded, space->size is 0 as well. This causes this chunk of code:

 if (space->size <= block_offset) {
  ulint actual_size;

  mutex_exit(&fil_system->mutex);
  fil_extend_space_to_desired_size(&actual_size, space->id,
       ((block_offset + 1) / 64 + 1) * 64);
  mutex_enter(&fil_system->mutex);
  /* should retry? but it may safe for xtrabackup for now. */
 }

to think this is a request past the end of the file and to grow the file to the 1MB boundary.

A simple fix is to also test if block_offset != 0 before growing the file. You could also check to see if space->size == 0 (tablespace hasn't been loaded). But that would be unnecessary because if space->size > 0, a request for block_offset 0 would fail the space->size <= block_offset test

Related branches

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Thank you for the problem report and the fix!

Changed in percona-xtrabackup:
status: New → Triaged
importance: Undecided → Medium
Stewart Smith (stewart)
tags: added: contribution
Revision history for this message
Alexey Kopytov (akopytov) wrote :

This is a rather serious problem in cases when there are large numbers of tablespaces. Raising the priority and we should fix it in 2.0.6.

Revision history for this message
Chris Boulton (chris.boulton) wrote :

I hadn't actually come across this until this evening, and we've only just managed to scrape by while preparing a backup (SST in PXC). We managed to go from disks usually 20% full, to 98% full, with our setup. We're now sitting at 97% full with the expanded tablespace fies.

Kinda funny that you just updated this issue today, Alexey - it's almost like you psychically saw me coming.

Are there any thoughts on Daniel's fix above, or if there's a better alternative? (it's something I'd like to fix ASAP and will probably end up rolling my own binaries for)

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Chris,

Daniel's fix looks correct.

Revision history for this message
Marc Castrovinci (marc-castrovinci) wrote :

Glad I found this bug. I thought I was going crazy when most my tables had larger idb files after a restore.

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-321

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Related questions