Use fallocate() when creating Whisper files on systems that support it

Bug #957827 reported by Michael Leinartas on 2012-03-17
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
whisper
Fix Committed
Medium
Michael Leinartas

Bug Description

As originally posted here: https://bugs.launchpad.net/graphite/+bug/885944

A better solution would be to detect the presence of 'fallocate' and use this if available. This will preserve data ordering and completely eliminate the excess I/O on database creation.

From the 'fallocate' man page:

DESCRIPTION
       fallocate is used to preallocate blocks to a file. For filesystems which support the fallocate system call, this
       is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This
       is much faster than creating a file by filling it with zeros.

       As of the Linux Kernel v2.6.31, the fallocate system call is supported by the btrfs, ext4, ocfs2, and xfs filesys-
       tems.

       The exit code returned by fallocate is 0 on success and 1 on failure.

Python support is available in linux-ftools. On a side note, this python package also adds POSIX FADVISE support, which is also useful; ie. POSIX_FADV_RANDOM to disable read-ahead, which can reduce trashing of the page cache and therefore boost I/O performance by only keeping useful metrics in the cache). POSIX_FADV_WILLNEED can also be used to prefetch only the metrics that will be needed (start, end file offset).

http://code.google.com/p/linux-ftools/

Sidnei da Silva (sidnei) on 2012-05-08
affects: graphite → whisper
Michael Leinartas (mleinartas) wrote :
Changed in whisper:
status: Confirmed → Fix Committed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers