Activity log for bug #1390472

Date Who What changed Old value New value Message
2014-11-07 13:23:59 Brian Cline bug added bug
2014-11-07 13:24:29 Brian Cline description We've seen a three-way split brain scenario crop up for a container database during a replication network NIC outage that occurred on one replica holder. Since the replication network is physically different, it was still able to accept requests from other servers, and continued processing them. However, when the NIC link came back up for that host, there somehow ended up being three different copies of that container database. After several weeks replication has not reconciled any of the different copies. This is a known issue that has cropped up before with several other major vendors, so documenting here. Working on a way to produce a test that can reliably reproduce for this issue. We've seen a three-way split brain scenario crop up for a container database during a replication network NIC outage that occurred on one replica holder. Since the replication network is physically different, it was still able to accept requests from proxy servers, and continued processing them. However, when the NIC link came back up for that host, there somehow ended up being three different copies of that container database. After several weeks replication has not reconciled any of the different copies. This is a known issue that has cropped up before with several other major vendors, so documenting here. Working on a way to produce a test that can reliably reproduce for this issue.
2014-11-07 13:26:42 Brian Cline swift: assignee Brian Cline (briancline)
2014-11-07 13:26:45 Brian Cline swift: status New Incomplete
2015-10-12 18:53:17 Brian Cline summary 3-way split brain during rpelication network NIC outage on one object-server 3-way split brain during NIC outage for dedicated replication network
2017-09-11 20:19:34 Tim Burke swift: status Incomplete Invalid