[SRU] "radosgw-admin bucket limit check" has duplicate entries if bucket count exceeds 1000 (max_entries)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Cloud Archive |
Fix Released
|
Undecided
|
Unassigned | ||
Ussuri |
Fix Released
|
Undecided
|
Unassigned | ||
ceph (Ubuntu) |
Fix Released
|
Medium
|
nikhil kshirsagar | ||
Focal |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
The "radosgw-admin bucket limit check" command has a bug in octopus.
Since we do not clear the bucket list in RGWRadosUser:
This bug is triggered if bucket count exceeds 1000 (default max_entries).
------
$ dpkg -l | grep ceph
ii ceph 15.2.12-
ii ceph-base 15.2.12-
ii ceph-common 15.2.12-
ii ceph-mds 15.2.12-
ii ceph-mgr 15.2.12-
ii ceph-mgr-
ii ceph-mon 15.2.12-
ii ceph-osd 15.2.12-
ii libcephfs2 15.2.12-
ii python3-
ii python3-ceph-common 15.2.12-
ii python3-cephfs 15.2.12-
$ sudo radosgw-admin bucket list | jq .[] | wc -l
5572
$ sudo radosgw-admin bucket limit check | jq .[].buckets[
20572
$ sudo radosgw-admin bucket limit check | jq '.[].buckets[] | select(
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
-------
Fix proposed through https:/
diff --git a/src/rgw/
index 2b7a313ed91.
--- a/src/rgw/
+++ b/src/rgw/
@@ -35,6 +35,7 @@ int RGWRadosUser:
RGWUserBuckets ulist;
bool is_truncated = false;
int ret;
+ buckets.clear();
ret = store->
-------
tested and verified the fix works:
$ sudo dpkg -l | grep ceph
ii ceph 15.2.14-
ii ceph-base 15.2.14-
ii ceph-common 15.2.14-
ii ceph-mds 15.2.14-
ii ceph-mgr 15.2.14-
ii ceph-mgr-
ii ceph-mon 15.2.14-
ii ceph-osd 15.2.14-
ii libcephfs2 15.2.14-
ii python3-
ii python3-ceph-common 15.2.14-
ii python3-cephfs 15.2.14-
ubuntu@
ceph:
Installed: 15.2.14-
Candidate: 15.2.14-
$ sudo radosgw-admin bucket list | jq .[] | wc -l
5572
$ sudo radosgw-admin bucket limit check | jq .[].buckets[
5572
$ sudo radosgw-admin bucket limit check | jq '.[].buckets[] | select(
{
"bucket": "bucket_1095",
"tenant": "",
"num_objects": 5,
"num_shards": 3,
"objects_
"fill_status": "OK"
}
----------
[Impact]
duplicated bucket name entries appear in the customers outputs when they script the `radosgw-admin bucket limit check` commands.
To reproduce:
Create more than 1000 (default value of max_entries) buckets in a cluster, and run 'radosgw-admin bucket limit check'
Duplicated entries are seen in the output on Octopus. For example,
$ sudo radosgw-admin bucket list | jq .[] | wc -l
5572
$ sudo radosgw-admin bucket limit check | jq .[].buckets[
20572
[Test case]
Create more than 1000 buckets in a cluster, then run the 'radosgw-admin bucket limit check' command. There should be no duplicated entries in the output. Below is correct output, where the numbers match.
$ sudo radosgw-admin bucket limit check | jq .[].buckets[
5572
$ sudo radosgw-admin bucket list | jq .[] | wc -l
5572
[Where problems could occur]
The duplicate entries could end up causing admins or even scripts to assume that there are more buckets than there really are.
[Other Info]
- The patch was provided by Nikhil Kshirsagar (attached here)
- Upstream tracker: https:/
- Upstream PR: https:/
- Patched into Octopus upstream release.
tags: | added: sts sts-sru-needed |
tags: | removed: sts-sru-needed |
description: | updated |
tags: | added: sts-sru-needed |
The attachment "0001-rgw- clear-buckets- before- calling- list_buckets. patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.
[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]