2.21-22:Issues with Cassandra Backup/Restore

[Sandeep Sridhar]

I have discussed this with Sudhee, Bhushana. Per Bhushan's suggestion, I am filing this bug for tracking.

Hi Sudhee,

  Thanks a lot for your help yesterday. Post our discussion, I spent sometime reviewing backup_restore.py code

I find two problems with the way we backup cassandra :

Issue 1:
In case of local backup, we execute cp –R /var/lib/cassandra/data ~/contrail_bkup_data/<hostname>. This command not only copies snapshot files but also SSTable files. I suppose SSTable* are not required during restore. Though we define skip_keyspace=[“ContrailAnalytics”], the SSTable files on ContrailAnalytics directory are still copied to the backup directory. The only difference is, in the backed up path, there won’t be any “snapshots” directory for ContrailAnalytics. This consumes huge disk space and bandwidth (~130G ). The other thing to check is whether scp works for data size of this capacity without bailing out.

SSTable* are deleted/created dynamically. If one file is merged into a new file, the old file gets deleted. When this happens during the cp command, we might try copying files that no longer exist and we finally exit with the errors I pointed out yesterday in the log files.

- Sandeep.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/18561
Submitter: aswani kumar (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/18561
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/d817ca8095c79634ffab1ea91ac4974d9e2b15a0
Submitter: Zuul
Branch: R2.21.x

commit d817ca8095c79634ffab1ea91ac4974d9e2b15a0
Author: Aswani Kumar Reddy G <email address hidden>
Date: Sat Mar 19 14:00:45 2016 +0530

Added fix to skip backup and restore instances for TSN nodes

Change-Id: I5f68cead62c39f5e5634e00716a1b5484f7e63fb
Closes-Bug: #1552174