Comment 10 for bug 1860200

Revision history for this message
Byron (byronester) wrote : Re: [Bug 1860200] Re: Add multi core support for compression

Ok, thanks. I don't think I fully understood that part of the conversation.
Thanks Ken and sorry for the confusion.
I think with all of these parallel compression/decompression libraries,
I've always seen an option to specify the level
of parallelism. MGZip for example has a threads parameter. If you force it
to use all cores, it will, but by no means do
you have to do that.

On Thu, 27 Feb 2020 at 00:25, Kenneth Loafman <email address hidden> wrote:

> Go back and read #3 again. I'm talking about running like this:
>
> duplicity--->backend process 1
> --->backend process 2
> ...
> --->backend process N
>
> Where each backend process handles gzip,gpg,plain as needed. This gives
> N-way multiprocessing and uses N cores for both IO and processing. In
> this scenario a multithreaded gzip would actually slow down the
> processing if it were to use all cores.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1860200
>
> Title:
> Add multi core support for compression
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> Duplicity compresses and decompresses with a single process atm.
> Performance can be increased by parallelizing this process. This in
> turn could be done using something like mgzip.
>
> https://code.launchpad.net/~byronester/duplicity/duplicity
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1860200/+subscriptions
>