Database replicator will rsync too often

Bug #1019712 reported by gholt on 2012-07-01
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Matthew Oliver

Bug Description

The database replicator will resort to "rsync+merge" mode too often. The code I noticed is in swift/common/

            # if the difference in rowids between the two differs by
            # more than 50%, rsync then do a remote merge.
            if rinfo['max_row'] / float(info['max_row']) < 0.5:

So, in the case where one database has 1 row, and the other 3 rows, an rsync+merge is done when it could've just sent the rows over.

Might not be a big deal as both methods should be pretty quick, but I thought dropping to rsync was less desirable. I'm not sure, so I figured I'd post a bug and let someone else take a look.

My thought is that it should only resort to rsync+merge when the difference is great (say, greater than per_diff * max_diff) and more than 50% different.

Chuck Thier (cthier) on 2012-12-04
Changed in swift:
status: New → Triaged
Simon Chang (changsimon) on 2013-09-27
Changed in swift:
assignee: nobody → Simon Chang (changsimon)
Simon Chang (changsimon) wrote :

I think part of the fix would be changing 0.5 to a smaller value, but not sure if we should just arbitrary choose some smaller value (say ... 0.2). The value should be based on production statistics.

Simon Chang (changsimon) on 2013-11-23
Changed in swift:
assignee: Simon Chang (changsimon) → nobody
drax (devesh-gupta) wrote :

Instead of checking on the percentage we can perform check on the difference.

Like if the difference between the rinfo['max_row'] and info['max_row'] is some threshold value we can perfrom rsync+merge.

Changed in swift:
assignee: nobody → drax (devesh-gupta)

Fix proposed to branch: master

Changed in swift:
status: Triaged → In Progress

Change abandoned by John Dickinson (<email address hidden>) on branch: master
Reason: Abandoning due to lack of activity since the last negative review. You can restore the change if you want to keep working on it.

John Dickinson (notmyname) wrote :

needs reverified

Changed in swift:
assignee: drax (devesh-gupta) → nobody
status: In Progress → Incomplete
Changed in swift:
assignee: nobody → Matthew Oliver (matt-0)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers