tar4ibd reads files inefficiently

[Alexey Kopytov]

tar4ibd is using very inefficient way to read files. To support compressed tables, it starts with a 1KB block size and calls read() with that size. If page verification fails, it increases the block size, calls lseek() to restore the original position and retries read().

It can be optimized in two ways (basically, by doing what xtrabackup binary does):

- read in large chunks, and then process them with different page sizes if necessary;
- don't consider small block sizes and default to 16KB when zip_size == 0, when the data file is not compressed.

I'm not sure it makes sense to fix this in 1.6, as tar4ibd is hopefully going away in 1.7. But reporting it just for the record.

[Stewart Smith (stewart)]

On Sun, 04 Dec 2011 14:53:09 -0000, Alexey Kopytov <email address hidden> wrote:
> Public bug reported:
>
> tar4ibd is using very inefficient way to read files. To support
> compressed tables, it starts with a 1KB block size and calls read() with
> that size. If page verification fails, it increases the block size,
> calls lseek() to restore the original position and retries read().
>
> It can be optimized in two ways (basically, by doing what xtrabackup
> binary does):
>
> - read in large chunks, and then process them with different page sizes if necessary;
> - don't consider small block sizes and default to 16KB when zip_size == 0, when the data file is not compressed.
>
> I'm not sure it makes sense to fix this in 1.6, as tar4ibd is hopefully
> going away in 1.7. But reporting it just for the record.

maybe posix_fadvise(WILLNEED) ?

--
Stewart Smith

[Alexey Kopytov (akopytov)]

On 05.12.11 3:57, Stewart Smith wrote:
> maybe posix_fadvise(WILLNEED) ?
>

I think the main problem here is not even caching, but rather 17
syscalls per page in the worst case, with the worst case being an
uncompressed tablespace.

[Alexey Kopytov (akopytov)]

It also breaks O_DIRECT reads for some filesystem. For example, it is known to fail for XFS created with -s size=4096.

[Stewart Smith (stewart)]

Closing as Won't Fix for the old 1.6 release. If a fix is needed for 1.6, please reopen here or contact us (Percona) to arrange it.

If this problem is seen in 2.0 or later, we can open a new bug for performance issues.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-1141