nodetool repair hangs repairing cassandra inconsistent tables

[wenqing liang]

2016-09-20 06:29:53:978158: root@vm-10:/opt/contrail/utils# fab -R database -- "nodetool repair"
[root@20.1.1.10] Executing task '<remainder>'
2016-09-20 06:30:16:965029: [root@20.1.1.10] run: nodetool repair
2016-09-20 06:30:16:965381: [root@20.1.1.10] out: [2016-09-20 06:30:19,747] Starting repair command #1, repairing 768 ranges for keyspace to_bgp_keyspace (parallelism=SEQUENTIAL, full=true)
2016-09-20 06:30:19:790918: [root@20.1.1.10] out:
2016-09-20 06:40:32:634253: [root@20.1.1.10] out:
2016-09-20 12:02:46:264219: [root@20.1.1.10] out:
2016-09-20 12:03:05:603164: [root@20.1.1.10] out:
2016-09-20 12:03:05:803621: [root@20.1.1.10] out: Killed
root@vm-10:/opt/contrail/utils#

This is due to https://bugs.launchpad.net/juniperopenstack/+bug/1610486 fix not in concert with 3.0.2.0 being upgrading from.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24317
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24325
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24326
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24326
Committed: http://github.org/Juniper/contrail-controller/commit/cac0637b0eba14b70842ba27eb730cd0c8d7fc2b
Submitter: Zuul
Branch: master

commit cac0637b0eba14b70842ba27eb730cd0c8d7fc2b
Author: Ignatious Johnson Christopher <email address hidden>
Date: Tue Sep 20 14:43:56 2016 -0700

Issue:
It is possible that the keyspace dir be empty without .db files,
in this case the cassa-db-restore.sh fails to copy .db from empty
dir and aborts and remaning keyspaces are not restored.

Fix:
Do copy/restore only if there are .db files precent in a table
directory of a keyspace.

Change-Id: I592560cfa7812ca70ba7ebca4408c8f2033c6278
Closes-Bug: 1625815

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24325
Committed: http://github.org/Juniper/contrail-controller/commit/4ba9054ac54bd9a10723561763f957eeceb370b2
Submitter: Zuul
Branch: R3.1

commit 4ba9054ac54bd9a10723561763f957eeceb370b2
Author: Ignatious Johnson Christopher <email address hidden>
Date: Tue Sep 20 14:43:56 2016 -0700

Issue:
It is possible that the keyspace dir be empty without .db files,
in this case the cassa-db-restore.sh fails to copy .db from empty
dir and aborts and remaning keyspaces are not restored.

Fix:
Do copy/restore only if there are .db files precent in a table
directory of a keyspace.

Change-Id: I592560cfa7812ca70ba7ebca4408c8f2033c6278
Closes-Bug: 1625815

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24317
Committed: http://github.org/Juniper/contrail-controller/commit/8a9a91fff81dd1ba575ae8bf0f239e19edf584f2
Submitter: Zuul
Branch: R3.0

commit 8a9a91fff81dd1ba575ae8bf0f239e19edf584f2
Author: Ignatious Johnson Christopher <email address hidden>
Date: Tue Sep 20 14:43:56 2016 -0700

Issue:
It is possible that the keyspace dir be empty without .db files,
in this case the cassa-db-restore.sh fails to copy .db from empty
dir and aborts and remaning keyspaces are not restored.

Fix:
Do copy/restore only if there are .db files precent in a table
directory of a keyspace.

Change-Id: I592560cfa7812ca70ba7ebca4408c8f2033c6278
Closes-Bug: 1625815