NFS backup_directory + purging is broken

[Andrew Garner]

NFS mounted with async does not immediately free the space on unlink(). I.e. unlink() does not block until the space is actually freed on the server and is instead freed in the background.

When paired with purge-policy=before-backup and estimation, the estimation part fails even if holland technically purged out enough data. NFS is still purging in the background when we calculate estimation so the space isn't actually freed.

I am not sure if there is a good solution here. Both 'rm -f' and python's shutil.rmtree give the same results. I was hoping perhaps a fileobj.truncate(0) + os.fsync() might work around this, but that does not appear to be the case. fsync does not block until this io is complete, unfortunately :( I suppose we could inject some (optional) sleep logic here. Maybe there's some better way to monitor that the unlink is actual complete. :-\