Use fallocate() when creating Whisper files on systems that support it
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
whisper |
Fix Committed
|
Medium
|
Michael Leinartas |
Bug Description
As originally posted here: https:/
A better solution would be to detect the presence of 'fallocate' and use this if available. This will preserve data ordering and completely eliminate the excess I/O on database creation.
From the 'fallocate' man page:
DESCRIPTION
fallocate is used to preallocate blocks to a file. For filesystems which support the fallocate system call, this
is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This
is much faster than creating a file by filling it with zeros.
As of the Linux Kernel v2.6.31, the fallocate system call is supported by the btrfs, ext4, ocfs2, and xfs filesys-
tems.
The exit code returned by fallocate is 0 on success and 1 on failure.
Python support is available in linux-ftools. On a side note, this python package also adds POSIX FADVISE support, which is also useful; ie. POSIX_FADV_RANDOM to disable read-ahead, which can reduce trashing of the page cache and therefore boost I/O performance by only keeping useful metrics in the cache). POSIX_FADV_WILLNEED can also be used to prefetch only the metrics that will be needed (start, end file offset).
affects: | graphite → whisper |
Fixed in https:/ /github. com/graphite- project/ whisper/ commit/ ee545cb09961d0e cad0ed8e0f37d3d 4039afead5