Add multi core support for compression
Bug #1860200 reported by
Byron
This bug affects 3 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
New
|
Undecided
|
Unassigned |
Bug Description
Duplicity compresses and decompresses with a single process atm. Performance can be increased by parallelizing this process. This in turn could be done using something like mgzip.
summary: |
- Add multi core support, possibly with mgzip + Add multi core support for compression |
Changed in duplicity: | |
assignee: | Kenneth Loafman (kenneth-loafman) → nobody |
importance: | Medium → Undecided |
milestone: | 0.9.00 → none |
status: | In Progress → New |
To post a comment you must log in.
We need to look into https:/ /facebook. github. io/zstd as it seems to be the up and coming multithreaded compressor. It is in use now in the Linux kernel and other major software. It should be stable and hopefully have a long life.
A proper Python interface lives at https:/ /pypi.python. org/pypi/ zstandard.
We would still have to support gzip decompression for a long while.