Some choices of delay_reaping & reclaim_age values can lead to container databases which are never deleted.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Confirmed
|
Medium
|
Unassigned |
Bug Description
If the delay_reaping value in the [account-reaper] section of account-server.conf is higher than the reclaim_age value of the [account-
then there exist the possiblity that the container databases of a deleted account are never deleted.
This is how it happens:
1. Account-reaper reaps the account, and thus sets the status of all container databases of the account to DELETED. Furthermore, the deleted_timestamp is adjusted, but NOT the reported_
2. Account-replicator runs and removes the account database.
3. Container-Updater runs, but cannot update the containers which are still on the disk, because the respective account database is no longer existent. The reported_
4. Container-
Step 3 & 4 are repeating forever. The container databases rest on the disk, and the account-server logs an error on step 3, because the account-server cannot find the account database. This error looks like this:
ERROR __call__ error with PUT /disk/807/
This bug is similar to #1332439. It was observed using openstack-swift 2.5.0 on centos7.
One userside workaround could be to adjust the interval of the container-updater (so that its clear it runs before the account-replicator which removes the account-database), or to set the delay_reaping value much lower than the reclaim_age of the account. At least there should be some documentation about this issue in the official docs. There is no
warning (for example in the configuration reference) of the interdependence of these two values.
summary: |
- Some choices of delay_reaping & reclaim_age values can lead to - unreclaimable container databases + Some choices of delay_reaping & reclaim_age values can lead to container + databases which are never deleted. |
description: | updated |
Why is the reported_ deleted_ timestamp not updated after a reaper run on a container database? In https:/ /github. com/openstack/ swift/blob/ master/ swift/container /server. py inside the method account_update, the
account db gets informed, that the container db has been deleted. Could we not update the container database after this step (via broker.reported)?