In the resize operation, during copying files from source to destination compute node  scp/rsync processes are not aborted after the instance is deleted because linux kernel doesn't delete instance files physically until all processes using the file handle is closed completely. Hence rsync/scp process keeps on running until it transfers 100% of file data.

One way to abort rsync/scp process is to call truncate command and set the size of the file to 0 bytes before deleting the instance files in the delete operation. We have added code to truncate files to 0 bytes and found out that it aborts rsync/scp process and doesn't copy the file to the destination node.

Previously rsync/scp copies 100% of file data but after adding truncate command, if the instance is deleted during copying files using rsync process it raises following ProcessExecutionError

ProcessExecutionError: Unexpected error while running command.
   Command: rsync --sparse --compress
/opt/stack/data/nova/instances/49225011-aefb-4262-b89c-50daa967fe98_resize/d
isk
10.69.4.130:/opt/stack/data/nova/instances/49225011-aefb-4262-b89c-50daa967f
e98/disk
   Exit code: 23
   Stdout: u''
   Stderr: u'rsync: read errors mapping
"/opt/stack/data/nova/instances/49225011-aefb-4262-b89c-50daa967fe98_resize/
disk": No data available (61)\nfile has vanished:
"/opt/stack/data/nova/instances/49225011-aefb-4262-b89c-50daa967fe98_resize/
disk"\nrsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1183) [sender=3.1.0]\n'


Also timing wise, we have observed rsync/scp processes are aborted early as compared to copying 100% of file but unfortunately the difference is not that big, it's merely 10-15% gain in time.

Results:

Following are the test results for rsync/scp processes for 25,50,75 and 100 GB disks.

*Command rsync, 25 GB*
Master Branch:
Processing Time: 9:40 mins
Source Node Files: Deleted
Destination Node Files: disk file of 25 GB present in instance directory

Truncate Patch:
Processing Time: 7:30 mins
Source Node Files: Deleted
Destination Node Files: empty instance directory




*Command scp, 25 GB*
Master Branch:
Processing Time: 5:11 mins
Source Node Files: Deleted
Destination Node Files: disk file of 25 GB present in instance directory

Truncate Patch:
Processing Time: 4:20 mins
Source Node Files: Deleted
Destination Node Files: disk file of 25 GB present in instance directory



*Command rsync, 50 GB*
Master Branch:
Processing Time: 17:23 mins
Source Node Files: Deleted
Destination Node Files: disk file of 50 GB present in instance directory

Truncate Patch:
Processing Time: 15:40 mins
Source Node Files: Deleted
Destination Node Files: empty instance directory



*Command scp, 50 GB*
Master Branch:
Processing Time: 11:25 mins
Source Node Files: Deleted
Destination Node Files: disk file of 50 GB present in instance directory

Truncate Patch:
Processing Time: 9:35 mins
Source Node Files: Deleted
Destination Node Files: disk file of 50 GB present in instance directory



*Command rsync, 75 GB*
Master Branch:
Processing Time: 25.53 mins
Source Node Files: Deleted
Destination Node Files: disk file of 75 GB present in instance directory

Truncate Patch:
Processing Time: 22:30 mins
Source Node Files: Deleted
Destination Node Files: empty instance directory



*Command scp, 75 GB*
Master Branch:
Processing Time: 16:24 mins
Source Node Files: Deleted
Destination Node Files: disk file of 75 GB present in instance directory

Truncate Patch:
Processing Time: 14:58 mins
Source Node Files: Deleted
Destination Node Files: disk file of 75 GB present in instance directory



*Command rsync, 100 GB*
Master Branch:
Processing Time: 35:15 mins
Source Node Files: Deleted
Destination Node Files: disk file of 100 GB present in instance directory

Truncate Patch:
Processing Time: 32:30 mins
Source Node Files: Deleted
Destination Node Files: empty instance directory


*Command scp, 100 GB*
Master Branch:
Processing Time: 23:45 mins
Source Node Files: Deleted
Destination Node Files: disk file of 100 GB present in instance directory

Truncate Patch:
Processing Time: 21:15 mins
Source Node Files: Deleted
Destination Node Files: disk file of 100 GB present in instance directory