eCryptfs does not sync in spite of setting O_SYNC

[Masayoshi Mizuma]

When I checked write() performance on eCryptfs (kernel:2.6.28-rc9), I found
that eCryptfs didn't sync though I set O_SYNC as open()'s option.
I think this is bug.

I checked write() performance below environment:
---------------------------------------------------------------------
KERNEL VARSION : linux-2.6.28-rc9
TEST DEVICE : /dev/sda1 (ext3)
MOUNT POINT : /ioperf
eCryptfs's MOUNT POINT : /ioperf/test
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[root@ibis ~]# df -T | grep /ioperf
/dev/sda1 ext3 10407720 154276 9728924 2% /ioperf
/ioperf/test
          ecryptfs 10407720 154276 9728924 2% /ioperf/test
[root@ibis ~]#
[root@ibis ~]# cat /etc/mtab | grep /ioperf
/dev/sda1 /ioperf ext3 rw 0 0
/ioperf/test /ioperf/test ecryptfs rw,ecryptfs_sig=a070a5801b4476a0,ecryptfs_cipher=aes,ecryptfs_key_bytes=16, 0 0
[root@ibis ~]#
[root@ibis ~]# /sbin/hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads: 1192 MB in 2.00 seconds = 596.08 MB/sec
 Timing buffered disk reads: 170 MB in 3.01 seconds = 56.50 MB/sec
[root@ibis ~]#
---------------------------------------------------------------------

I observed write()'s execution time (elapsed time) by using gettime() which
I created and used gettimeofday(). I set gettime() before write() and after
it, then I got the difference.

gettime()'s code is:
-----------------------------------------------------------
double gettime ()
{
        struct timeval tv;
        gettimeofday(&tv, NULL);
        return tv.tv_sec + (double)tv.tv_usec*1e-6;
}
-----------------------------------------------------------

how to observe execution time:
-----------------------------------------------------------
before_time = gettime();
write(fd, WRITE_SIZE);
after_time = gettime();

result = after_time - before_time;
-----------------------------------------------------------

I show you two tables which show write()'s perfrmance result on eCryptfs below:

    table1: Result with O_SYNC as open()'s option.
    table2: Result without O_SYNC as open()'s option.

each table has two columns:

write size (as write()'s option) | write()'s execution time (elapsed time)

table1:
Result with O_SYNC [open(O_WRONLY|O_CREAT|O_SYNC, 0644)]

write size (kb) | execution time (second)
-----------------------------------------
1 | 0.000060
2 | 0.000063
4 | 0.000061
8 | 0.000172
16 | 0.000378
32 | 0.000787
64 | 0.001615
128 | 0.003212
256 | 0.006548
512 | 0.013037
1024 | 0.026057
2048 | 0.055292
4096 | 0.105369
8192 | 0.208334
16384 | 0.418020
32768 | 0.819937
65536 | 1.684167
131072 | 3.416073
-----------------------------------------

table2:
Result without O_SYNC [open(O_WRONLY|O_CREATE, 0644)]

write size (kb) | execution time (second)
-----------------------------------------
1 | 0.000057
2 | 0.000058
4 | 0.000059
8 | 0.000171
16 | 0.000374
32 | 0.000776
64 | 0.001605
128 | 0.003244
256 | 0.006664
512 | 0.012978
1024 | 0.025892
2048 | 0.054889
4096 | 0.105458
8192 | 0.208748
16384 | 0.412646
32768 | 0.826263
65536 | 1.698859
131072 | 3.455355
-----------------------------------------

Comparing table1 to table2, we can found that the execution time is almost
the same.
If we set O_SYNC as open()'s option, sync working (writeback to disk) should be
called, so table1's execution time should slow than table2's. However, each
table's execution time is almost the same, so I think that sync does not called.

I think this is bug. Please check this.

Regards,
Masasyoshi Mizuma

[Tyler Hicks (tyhicks)]

Thanks for the bug report. Your assumptions are correct, the O_SYNC flag is currently ignored. It looks like O_RDWR and O_LARGEFILE are the only flags used when opening the lower file. This is due to the fact that we have a single, persistent lower file that gets opened. We are planning some upcoming design changes which will affect this area and may allow eCryptfs to support O_SYNC.

[Tyler Hicks (tyhicks)]

For now, I'm going to mark this won't fix. I'd like this functionality, but it would be a considerable undertaking and is low on my importance list. If someone sends a nicely written kernel patch to lkml to support this, I'll be sure to ack it.

[Dustin Kirkland  (kirkland)]

We've changed this from won't fix to Low/Wishlist. This is a design issue that will take a non-trivial amount of effort to fix. It should be fixed. It just probably won't happen any time soon. Stay tuned. Thanks for the report.

:-Dustin

[Jason Xing (wlxing)]

It has been a long time...It seems that nobody has fixed that problem yet.

So I'm wondering that is someone working on that or does someone go any further? If not, I decide to solve that problem out of interests.

Any advice are welcome : - )

Jason