shard audit exaggerates reports of overlapping shard ranges

[Alistair Coles]

When sharder audits a root container and finds overlapping ranges, there is a tendency for each overlap to be logged more than once. For, example the following might be logged:

Audit failed for root var/folders/n4/nf8krx110f39llj82fbw_xjm0000gp/T/tmpFbb92K/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db (a/c): overlapping ranges in state created: s-z k-t, overlapping ranges in state created: k-t s-z

This can be illustrated by changing a unit test to assert that both variations of the same overlap are logged:

diff --git a/test/unit/container/test_sharder.py b/test/unit/container/test_sharder.py
index 22cd86797..7df1d145b 100644
--- a/test/unit/container/test_sharder.py
+++ b/test/unit/container/test_sharder.py
@@ -4397,6 +4397,9 @@ class TestSharder(BaseTestSharder):
             self.assertIn(
                 'overlapping ranges in state %s: k-t s-z' % state_text,
                 line)
+ self.assertIn(
+ 'overlapping ranges in state %s: s-z k-t' % state_text,
+ line)

         expected_stats = {'attempted': 1, 'success': 0, 'failure': 1}
         shard_bounds = (('a', 'j'), ('k', 't'), ('s', 'z'))

(based on commit 257041aff)

[Matthew Oliver (matt-0)]

This is because your ShardRange.__lt__ isn't sensitive to overlaps. Just whole range differences. Ie. self.upper < other.lower. Which doesn't work in an overlap

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/swift/+/791485

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/swift/+/791485
Committed: https://opendev.org/openstack/swift/commit/1b183f221d258819c82ee58961afc765529314ac
Submitter: "Zuul (22348)"
Branch: master

commit 1b183f221d258819c82ee58961afc765529314ac
Author: Alistair Coles <email address hidden>
Date: Fri May 14 13:59:37 2021 +0100

    sharder: report perfectly overlapping shard ranges

    Previously, shard ranges that overlapped perfectly (i.e. their bounds
    were equal) were erroneously excluded from overlapping shard reports.
    With this patch they will be reported in logs and cause shard audit
    failures.

    Also, the format of the audit log line is simplified to avoid
    unnecessary duplication of information.

    Change-Id: Ie15c9f40a132374c89337f0009fb5cf5a8e62c51
    Closes-Bug: #1928459
    Related-Bug: #1913332