As follows from the log, the node was missing a row to update. Assuming that all nodes but one had this situation it is logical to assume that the remaining node was inconsistent with the rest of the cluster.
It could happen for many reasons and arbitrarily long ago. But the most likely cause is human error when restarting the cluster. If you could carefully recall EXACTLY how you did it you may find a clue.
Restarting http servers and ha-proxy load balancing is not a CAUSE of inconsistency, unless there is some nasty bug. Such bugs proved to be rather rare (1 or 2 through the history), so you should look into how you restarted the nodes first. ha-proxy balancing could CONTRIBUTE to it if you start the nodes in wrong order.
Hi Jason,
As follows from the log, the node was missing a row to update. Assuming that all nodes but one had this situation it is logical to assume that the remaining node was inconsistent with the rest of the cluster.
It could happen for many reasons and arbitrarily long ago. But the most likely cause is human error when restarting the cluster. If you could carefully recall EXACTLY how you did it you may find a clue.
Restarting http servers and ha-proxy load balancing is not a CAUSE of inconsistency, unless there is some nasty bug. Such bugs proved to be rather rare (1 or 2 through the history), so you should look into how you restarted the nodes first. ha-proxy balancing could CONTRIBUTE to it if you start the nodes in wrong order.