OpenStack Object Storage (Swift)

Database replicator will rsync too often

Reported by gholt on 2012-07-01
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)

Bug Description

The database replicator will resort to "rsync+merge" mode too often. The code I noticed is in swift/common/

            # if the difference in rowids between the two differs by
            # more than 50%, rsync then do a remote merge.
            if rinfo['max_row'] / float(info['max_row']) < 0.5:

So, in the case where one database has 1 row, and the other 3 rows, an rsync+merge is done when it could've just sent the rows over.

Might not be a big deal as both methods should be pretty quick, but I thought dropping to rsync was less desirable. I'm not sure, so I figured I'd post a bug and let someone else take a look.

My thought is that it should only resort to rsync+merge when the difference is great (say, greater than per_diff * max_diff) and more than 50% different.

Chuck Thier (cthier) on 2012-12-04
Changed in swift:
status: New → Triaged
Simon Chang (changsimon) on 2013-09-27
Changed in swift:
assignee: nobody → Simon Chang (changsimon)
Simon Chang (changsimon) wrote :

I think part of the fix would be changing 0.5 to a smaller value, but not sure if we should just arbitrary choose some smaller value (say ... 0.2). The value should be based on production statistics.

Simon Chang (changsimon) on 2013-11-23
Changed in swift:
assignee: Simon Chang (changsimon) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers