ALL_O_DIRECT causes unaligned AIO/DIO

Bug #1033051 reported by Raghavendra D Prabhu on 2012-08-04
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Fix Released
Medium
Hrvoje Matijakovic
5.1
Fix Released
Medium
Hrvoje Matijakovic
5.5
Fix Released
Medium
Hrvoje Matijakovic

Bug Description

Setting innodb_flush_method to ALL_O_DIRECT seems to be causing unaligned AIO/DIO.

It causes "EXT4-fs (dm-0): Unaligned AIO/DIO on inode 1439824 by mysqld; performance will be poor." in dmesg where inode 1439824 is none other than ib_logfile0.

To reproduce:

1. Set innodb_flush_method=ALL_O_DIRECT in my.cnf and restart the server.

2. Check for something along the lines of

"EXT4-fs (dm-0): Unaligned AIO/DIO on inode 1442673 by mysqld; performance will be poor."

in dmesg.

Now, this dmesg message is from http://lxr.linux.no/#linux+v3.5/fs/ext4/file.c#L129 and is printed when unaligned AIO (with direct-io) take place; it is serialized to avoid the corruption, and it is this serialization which causes performance loss.

Since this message appears only on ALL_O_DIRECT and not on O_DIRECT, it looks like somewhere in log flushing/write code there must be unaligned access (aligned I/O is a must for direct-io).

Related branches

tags: added: i25299

Wasn't that hard to diagnose after checking log_group_write_buf that writes are aligned wrt. OS_FILE_LOG_BLOCK_SIZE. This block size is tunable only in Percona server with innodb-log-block-size.

So, the 'fix' is quite simply setting innodb-block-size to 4096.

4096 because it is the default log-block-size in ext4 and xfs.

I guess there is no code fix required here (since there is no bug), however, a note can be added in innodb-log-block-size's documentation about this dmesg warning and/or about the crash https://bugs.launchpad.net/percona-server/+bug/1040803 and the workaround.

tags: added: doc
Stewart Smith (stewart) wrote :

Was innodb-log-block-size set to get this warning? if we get this warning by default then we may want to look at fixing something.

Jay Janssen (jay-janssen) wrote :

I don't believe innodb_log_block_size was being set, but this was so long ago that I have forgotten even which customer it was.

@Stewart,

From what I have seen, innodb-log-block-size was not set during
this, and that is what caused this, since default
innodb-log-block-size is 512bytes whereas the filesystem block-size
is 4k and ext* filesystems print this warning to dmesg (xfs
doesn't print but may not be affected as much as well) due to
non-alignment.

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1266

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers