Ok, thanks. I don't think I fully understood that part of the conversation.
Thanks Ken and sorry for the confusion.
I think with all of these parallel compression/decompression libraries,
I've always seen an option to specify the level
of parallelism. MGZip for example has a threads parameter. If you force it
to use all cores, it will, but by no means do
you have to do that.
On Thu, 27 Feb 2020 at 00:25, Kenneth Loafman <email address hidden> wrote:
> Go back and read #3 again. I'm talking about running like this:
>
> duplicity--->backend process 1
> --->backend process 2
> ...
> --->backend process N
>
> Where each backend process handles gzip,gpg,plain as needed. This gives
> N-way multiprocessing and uses N cores for both IO and processing. In
> this scenario a multithreaded gzip would actually slow down the
> processing if it were to use all cores.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1860200
>
> Title:
> Add multi core support for compression
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> Duplicity compresses and decompresses with a single process atm.
> Performance can be increased by parallelizing this process. This in
> turn could be done using something like mgzip.
>
> https://code.launchpad.net/~byronester/duplicity/duplicity
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1860200/+subscriptions
>
Ok, thanks. I don't think I fully understood that part of the conversation. decompression libraries,
Thanks Ken and sorry for the confusion.
I think with all of these parallel compression/
I've always seen an option to specify the level
of parallelism. MGZip for example has a threads parameter. If you force it
to use all cores, it will, but by no means do
you have to do that.
On Thu, 27 Feb 2020 at 00:25, Kenneth Loafman <email address hidden> wrote:
> Go back and read #3 again. I'm talking about running like this: -->backend process 1 /bugs.launchpad .net/bugs/ 1860200 /code.launchpad .net/~byroneste r/duplicity/ duplicity /bugs.launchpad .net/duplicity/ +bug/1860200/ +subscriptions
>
> duplicity-
> --->backend process 2
> ...
> --->backend process N
>
> Where each backend process handles gzip,gpg,plain as needed. This gives
> N-way multiprocessing and uses N cores for both IO and processing. In
> this scenario a multithreaded gzip would actually slow down the
> processing if it were to use all cores.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> Add multi core support for compression
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> Duplicity compresses and decompresses with a single process atm.
> Performance can be increased by parallelizing this process. This in
> turn could be done using something like mgzip.
>
> https:/
>
> To manage notifications about this bug go to:
> https:/
>