os.system() forks duplicity, doubles memory usage
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
Fix Released
|
Medium
|
Unassigned |
Bug Description
It took a bit of a journey to arrive at this particular bug, so please bear with me on the following sad story.
1. Backups kept dying because `duplicity-
2. Went completely down the rabbit hole and found via strace that immediately before the failure there's a failed `clone()` call with `1 ENOMEM (Cannot allocate memory)`. [note: we have fat backups so *one* duplicity process uses most of the memory, and doubling that is just out of the question]
3. Dug into the source and found that there's literally *one* `os.fork()` call, and that's in duplicity/
4. Spun up a duplicity backup with strace on a non-broken server and watched for what the fork actually *did*. Turns out it does `execve('sh', '-c', 'cp ...')` and are you flippin kidding me?
5. Checked in with #python and confirmed that os.system() causes a fork and got quite a bit of sass on your behalf for using it at all.
6. Monkey-patched production, and lo and behold the backup completes!
= Reproduction:
Run a backup with `--no-encryption`.
Gzip and GPG modes don't invoke this particular flavor of problem, though the aforementioned fork in gpginterface.py likely isn't doing you guys any favors.
= Fix:
Replace duplicity/
os.system("cp -p \"%s\" \"%s\"" % (src.name, tgt.name))
With:
from shutil import copyfile
copyfile(
That's literally it and you've halved your memory footprint at the end of the run.
= Errata:
- duplicity versions: 0.7.10 and 0.7.12
- OS: Centos 7.2
- FS: ext4
Changed in duplicity: | |
importance: | Undecided → Medium |
milestone: | none → 0.7.13 |
status: | New → Fix Committed |
Changed in duplicity: | |
status: | Fix Committed → Fix Released |