Go back and read #3 again. I'm talking about running like this:
duplicity--->backend process 1 --->backend process 2
... --->backend process N
Where each backend process handles gzip,gpg,plain as needed. This gives N-way multiprocessing and uses N cores for both IO and processing. In this scenario a multithreaded gzip would actually slow down the processing if it were to use all cores.
Go back and read #3 again. I'm talking about running like this:
duplicity- -->backend process 1
--->backend process 2
--->backend process N
...
Where each backend process handles gzip,gpg,plain as needed. This gives N-way multiprocessing and uses N cores for both IO and processing. In this scenario a multithreaded gzip would actually slow down the processing if it were to use all cores.