fail when os_file_set_nocache fails

[Mark Callaghan]

With innodb_flush_method == ALL_O_DIRECT or O_DIRECT, os_file_set_nocache is called after the file has been opened to put the fd in direct IO mode. os_file_set_nocache logs an error but does not return an error on failure. XtraDB appears to not call fsync when the innodb_io_patches.patch has been applied (similar behavior is in the Facebook patch). So, XtraDB and the Facebook patch are not crash safe when os_file_set_nocache fails. I have a pending change to crash InnoDB when that happens. The alternative is to detect that failure and switch to buffered IO.

[Yasufumi Kinoshita (yasufumi-kinoshita)]

if O_DIRECT is set, InnoDB tries to use TABLESPACE files in O_DIRECT.
And in log_checkpoint(),
 if (srv_unix_file_flush_method != SRV_UNIX_NOSYNC) {
  fil_flush_file_spaces(FIL_TABLESPACE);
 }
so, tablespace files seem to be flushed even if O_DIRECT is fail.

The problem seems only about ALL_O_DIRECT of XtraDB.

So, how about add following code to the fail case of os_file_set_nocache() ?

==============
if (srv_unix_file_flush_method == SRV_UNIX_ALL_O_DIRECT) {
        srv_unix_file_flush_method = SRV_UNIX_O_DIRECT;
}
==============

[Stewart Smith (stewart)]

from open(2)

              The O_DIRECT flag on its own makes at an effort to transfer data
              synchronously, but does not give the guarantees of the O_SYNC
              that data and necessary metadata are transferred. To guarantee
              synchronous I/O the O_SYNC must be used in addition to O_DIRECT.
              See NOTES below for further discussion.

so both XtraDB and the facebook patches are technically incorrect, although they may just happen to work on most kernels/filesystem combinations there is no reason for them to continue to do so.

[Alexey Kopytov (akopytov)]

This is certainly not a critical bug, as we still call fsync() on datafiles even if setting O_DIRECT fails. The only thing that needs to be fixed here is a very specific case when the log files are stored on tmpfs :) In this case we should downgrade SRV_UNIX_ALL_O_DIRECT to SRV_UNIX_O_DIRECT so fsync() is used on the log files as well.

[Nickolay Ihalainen (ihanick)]

https://github.com/percona/percona-server/pull/1230, https://github.com/percona/percona-server/pull/1231, https://github.com/percona/percona-server/pull/1232

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1205