diskfile: don't remove recently written non-durables
DiskFileManager will remove any stale files during
cleanup_ondisk_files(): these include tombstones and nondurable EC
data fragments whose timestamps are older than reclaim_age. It can
usually be safely assumed that a non-durable data fragment older than
reclaim_age is not going to become durable. However, if an agent PUTs
objects with specified older X-Timestamps (for example the reconciler
or container-sync) then there is a window of time during which the
object server has written an old non-durable data file but has not yet
committed it to make it durable.
Previously, if another process (for example the reconstructor) called
cleanup_ondisk_files during this window then the non-durable data file
would be removed. The subsequent attempt to commit the data file would
then result in a traceback due to there no longer being a data file to
rename, and of course the data file is lost.
This patch modifies cleanup_ondisk_files to not remove old, otherwise
stale, non-durable data files that were only written to disk in the
preceding 'commit_window' seconds. 'commit_window' is configurable for
the object server and defaults to 60.0 seconds.
Reviewed: https:/ /review. opendev. org/c/openstack /swift/ +/800974 /opendev. org/openstack/ swift/commit/ bbaed18e9b681ce 9cf26ffa6a5d529 2f5cb219b7
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit bbaed18e9b681ce 9cf26ffa6a5d529 2f5cb219b7
Author: Alistair Coles <email address hidden>
Date: Thu Jul 15 18:08:01 2021 +0100
diskfile: don't remove recently written non-durables
DiskFileManager will remove any stale files during ondisk_ files() : these include tombstones and nondurable EC
cleanup_
data fragments whose timestamps are older than reclaim_age. It can
usually be safely assumed that a non-durable data fragment older than
reclaim_age is not going to become durable. However, if an agent PUTs
objects with specified older X-Timestamps (for example the reconciler
or container-sync) then there is a window of time during which the
object server has written an old non-durable data file but has not yet
committed it to make it durable.
Previously, if another process (for example the reconstructor) called ondisk_ files during this window then the non-durable data file
cleanup_
would be removed. The subsequent attempt to commit the data file would
then result in a traceback due to there no longer being a data file to
rename, and of course the data file is lost.
This patch modifies cleanup_ ondisk_ files to not remove old, otherwise
stale, non-durable data files that were only written to disk in the
preceding 'commit_window' seconds. 'commit_window' is configurable for
the object server and defaults to 60.0 seconds.
Closes-Bug: #1936508 249fb7b17bd5f41 9ffdaa616c0 77a63713e6ff8d0 fd3b6144f13
Related-Change: I0d519ebaaade35
Change-Id: I5f3318a44af64b