| 2017-09-13 21:42:31 |
Yann Collet |
bug |
|
|
added bug |
| 2017-09-13 22:02:33 |
Graham Inggs |
bug |
|
|
added subscriber Graham Inggs |
| 2017-09-20 15:14:31 |
Graham Inggs |
summary |
Please backport libzstd 1.3.1 from debian |
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful |
|
| 2017-09-20 15:17:32 |
Graham Inggs |
description |
Please backport libzstd 1.3.1 from debian to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
You can test-build the backport in your PPA with backportpackage:
$ backportpackage -u ppa:<lp username>/<ppa name> -s artful -d xenial libzstd
* xenial:
[ ] Package builds without modification
[ ] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[ ] libzstd-dev installs cleanly and runs
[ ] libzstd1 installs cleanly and runs
No reverse dependencies
|
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[ ] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[ ] libzstd-dev installs cleanly and runs
[ ] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-09-21 22:27:52 |
Yann Collet |
description |
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[ ] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[ ] libzstd-dev installs cleanly and runs
[ ] libzstd1 installs cleanly and runs
No reverse dependencies |
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-09-21 22:44:12 |
Graham Inggs |
xenial-backports: status |
New |
Confirmed |
|
| 2017-12-05 20:52:09 |
Dan Streetman |
bug |
|
|
added subscriber Dan Streetman |
| 2017-12-06 20:20:09 |
Andreas Hasenack |
bug |
|
|
added subscriber Andreas Hasenack |
| 2017-12-06 21:49:22 |
David Britton |
bug |
|
|
added subscriber David Britton |
| 2017-12-07 17:52:16 |
Andreas Hasenack |
bug task added |
|
libzstd (Ubuntu) |
|
| 2017-12-07 17:52:26 |
Andreas Hasenack |
libzstd (Ubuntu): status |
New |
Confirmed |
|
| 2017-12-07 17:52:57 |
Andreas Hasenack |
bug task deleted |
libzstd (Ubuntu) |
|
|
| 2017-12-07 17:53:28 |
Andreas Hasenack |
affects |
xenial-backports |
libzstd (Ubuntu) |
|
| 2017-12-07 17:54:27 |
Andreas Hasenack |
description |
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Impact]
* An explanation of the effects of the bug on users and
* justification for backporting the fix to the stable release.
* In addition, it is helpful, but not required, to include an
explanation of how the upload fixes this bug.
[Test Case]
* detailed instructions how to reproduce the bug
* these should allow someone who is not familiar with the affected
package to reproduce the bug and verify that the updated package fixes
the problem.
[Regression Potential]
* discussion of how regressions are most likely to manifest as a result of this change.
* It is assumed that any SRU candidate patch is well-tested before
upload and has a low overall risk of regression, but it's important
to make the effort to think about what ''could'' happen in the
event of a regression.
* This both shows the SRU team that the risks have been considered,
and provides guidance to testers in regression-testing the SRU.
[Other Info]
* Anything else you think is useful to include
* Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
* and address these questions in advance
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-12-07 17:56:18 |
Andreas Hasenack |
nominated for series |
|
Ubuntu Zesty |
|
| 2017-12-07 17:56:18 |
Andreas Hasenack |
nominated for series |
|
Ubuntu Xenial |
|
| 2017-12-07 17:56:40 |
David Britton |
bug task added |
|
libzstd (Ubuntu Xenial) |
|
| 2017-12-07 17:56:48 |
David Britton |
bug task added |
|
libzstd (Ubuntu Zesty) |
|
| 2017-12-07 20:30:27 |
Yann Collet |
description |
[Impact]
* An explanation of the effects of the bug on users and
* justification for backporting the fix to the stable release.
* In addition, it is helpful, but not required, to include an
explanation of how the upload fixes this bug.
[Test Case]
* detailed instructions how to reproduce the bug
* these should allow someone who is not familiar with the affected
package to reproduce the bug and verify that the updated package fixes
the problem.
[Regression Potential]
* discussion of how regressions are most likely to manifest as a result of this change.
* It is assumed that any SRU candidate patch is well-tested before
upload and has a low overall risk of regression, but it's important
to make the effort to think about what ''could'' happen in the
event of a regression.
* This both shows the SRU team that the risks have been considered,
and provides guidance to testers in regression-testing the SRU.
[Other Info]
* Anything else you think is useful to include
* Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
* and address these questions in advance
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases
* unpack, compile (make)
* compress a file with upstream version
* try to decompress with Ubuntu LTS 16.04 version -> it fails
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-12-08 11:53:32 |
Gianfranco Costamagna |
bug |
|
|
added subscriber LocutusOfBorg |
| 2017-12-08 20:12:45 |
Andreas Hasenack |
libzstd (Ubuntu Xenial): assignee |
|
Andreas Hasenack (ahasenack) |
|
| 2017-12-08 20:12:47 |
Andreas Hasenack |
libzstd (Ubuntu Zesty): assignee |
|
Andreas Hasenack (ahasenack) |
|
| 2017-12-08 20:12:51 |
Andreas Hasenack |
libzstd (Ubuntu Xenial): status |
New |
In Progress |
|
| 2017-12-08 20:12:55 |
Andreas Hasenack |
libzstd (Ubuntu Zesty): status |
New |
In Progress |
|
| 2017-12-08 20:13:12 |
Andreas Hasenack |
libzstd (Ubuntu): status |
Confirmed |
Fix Released |
|
| 2017-12-18 19:44:17 |
Andreas Hasenack |
description |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases
* unpack, compile (make)
* compress a file with upstream version
* try to decompress with Ubuntu LTS 16.04 version -> it fails
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases
* unpack, compile (make)
* compress a file with upstream version
* try to decompress with Ubuntu LTS 16.04 version -> it fails
Second test:
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-12-18 20:04:45 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/335340 |
|
| 2017-12-18 20:11:00 |
Andreas Hasenack |
merge proposal linked |
|
https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/335341 |
|
| 2017-12-20 06:36:26 |
Christian Ehrhardt |
bug |
|
|
added subscriber ChristianEhrhardt |
| 2017-12-26 13:20:28 |
Andreas Hasenack |
description |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases
* unpack, compile (make)
* compress a file with upstream version
* try to decompress with Ubuntu LTS 16.04 version -> it fails
Second test:
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases
* unpack, compile (make)
* compress a file with upstream version
* try to decompress with Ubuntu LTS 16.04 version -> it fails
Second test:
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-12-26 13:20:49 |
Andreas Hasenack |
description |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases
* unpack, compile (make)
* compress a file with upstream version
* try to decompress with Ubuntu LTS 16.04 version -> it fails
Second test:
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2017-12-26 13:21:34 |
Andreas Hasenack |
description |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
[Test Case]
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2018-01-04 17:57:44 |
Robie Basak |
bug |
|
|
added subscriber Robie Basak |
| 2018-01-17 12:29:01 |
Andreas Hasenack |
attachment added |
|
zstd-xenial.txt https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038187/+files/zstd-xenial.txt |
|
| 2018-01-17 12:32:39 |
Andreas Hasenack |
attachment added |
|
zstd-artful.txt https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038189/+files/zstd-artful.txt |
|
| 2018-01-17 13:46:57 |
Andreas Hasenack |
nominated for series |
|
Ubuntu Artful |
|
| 2018-01-17 13:47:10 |
Andreas Hasenack |
nominated for series |
|
Ubuntu Bionic |
|
| 2018-01-17 13:47:37 |
Andreas Hasenack |
libzstd (Ubuntu Zesty): status |
In Progress |
Won't Fix |
|
| 2018-01-17 14:05:14 |
Robie Basak |
bug task added |
|
libzstd (Ubuntu Artful) |
|
| 2018-01-17 14:07:34 |
Andreas Hasenack |
libzstd (Ubuntu Artful): status |
New |
Confirmed |
|
| 2018-01-17 14:07:38 |
Andreas Hasenack |
libzstd (Ubuntu Artful): status |
Confirmed |
In Progress |
|
| 2018-01-17 14:07:41 |
Andreas Hasenack |
libzstd (Ubuntu Artful): assignee |
|
Andreas Hasenack (ahasenack) |
|
| 2018-01-17 14:07:44 |
Andreas Hasenack |
libzstd (Ubuntu): status |
Fix Released |
In Progress |
|
| 2018-01-17 14:07:54 |
Andreas Hasenack |
libzstd (Ubuntu): assignee |
|
Andreas Hasenack (ahasenack) |
|
| 2018-01-17 19:13:58 |
Andreas Hasenack |
attachment added |
|
zstd-xenial.txt https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038387/+files/zstd-xenial.txt |
|
| 2018-01-17 19:14:31 |
Andreas Hasenack |
attachment added |
|
zstd-artful.txt https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038388/+files/zstd-artful.txt |
|
| 2018-01-17 19:43:04 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/336258 |
|
| 2018-01-17 19:45:11 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/336259 |
|
| 2018-01-17 19:46:07 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/336260 |
|
| 2018-01-24 12:41:45 |
Chris J Arges |
bug task added |
|
libzstd (Ubuntu Bionic) |
|
| 2018-02-02 12:32:06 |
Andreas Hasenack |
attachment added |
|
xenial-desktop-zstd-prompt.png https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5047557/+files/xenial-desktop-zstd-prompt.png |
|
| 2018-02-02 17:19:35 |
Robie Basak |
description |
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Special SRU Notes]
The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path.
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2018-02-02 19:08:15 |
Andreas Hasenack |
description |
[Special SRU Notes]
The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path.
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Special SRU Notes]
The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path.
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
There are three major test cases: compression tests, package upgrade tests, and release upgrade tests.
This SRU has so many tests because it's introducing a new package and a carefully thought out upgrade plan.
a) compression tests
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
b) package upgrade tests
b.1)XENIAL
* Install the xenial original packages:
$ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1
* Enable xenial-proposed and verify that a dist-upgrade will upgrade zstd, pull in the new libzstd1, and leave libzstd-dev alone:
$ sudo apt dist-upgrade
...
The following NEW packages will be installed:
libzstd1
The following packages will be upgraded:
zstd
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
* In the end you will have these installed, at these versions:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
* Installing the new versioned dev package should remove the unversioned one:
$ sudo apt install libzstd1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
libzstd-dev
The following NEW packages will be installed:
libzstd1-dev
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
* The end result being:
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
* Conversely, installing the unversioned dev package removes the versioned one::
Installing libzstd-dev removes libzstd1-dev:
$ sudo apt install libzstd-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
libzstd1-dev
The following NEW packages will be installed:
libzstd-dev
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
* Start from scratch and reinstall the original xenial zstd packages:
$ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1
* Verify that a simple apt upgrade (not dist-upgrade) upgrades zstd and installs the new libzstd1:
$ sudo apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
libzstd1
The following packages will be upgraded:
zstd
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 392 kB of archives.
After this operation, 904 kB of additional disk space will be used.
Do you want to continue? [Y/n]
* After the upgrade, you should have:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
b.2) ARTFUL
* starting with all original artful zstd packages installed:
ii libzstd-dev 1.3.1+dfsg-1
ii libzstd1 1.3.1+dfsg-1
ii zstd 1.3.1+dfsg-1
* a dist-upgrade upgrades them just fine, with no new dependencies introduced:
The following packages will be upgraded:
libzstd-dev libzstd1 zstd
3 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
* which results in:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
* libzstd1-dev, the transitional package, can also be installed just fine:
$ sudo apt install libzstd1-dev
The following NEW packages will be installed:
libzstd1-dev
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
* removing libzstd-dev also removes libzstd1-dev:
The following packages will be REMOVED:
libzstd-dev* libzstd1-dev*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
* installing just the new libzstd1-dev pull in the unversioned dev package:
$ sudo apt install libzstd1-dev
The following additional packages will be installed:
libzstd-dev libzstd1
The following NEW packages will be installed:
libzstd-dev libzstd1 libzstd1-dev
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
c) release upgrade tests
RELEASE UPGRADE FROM XENIAL TO ARTFUL
1) starting with the 0.5.1 dev package in xenial:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii zstd 0.5.1-1
dist-upgrade to proposed version:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
in the do-release-upgrade upgrade list:
libzstd-dev libzstd1 zstd
deemed obsolete and removed in the end: libzstd0
end result in artful:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
2) starting with the 1.3.1 dev package in xenial:
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
do-release-upgrade stats:
install list: libzstd-dev
upgrade list: libzstd1 libzstd1-dev zstd
remove (auto installed): libzstd0
end result:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
RELEASE UPGRADE FROM ARTFUL TO BIONIC
- with all zstd packages installed:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
do-release-upgrade stats:
upgrade list: libzstd-dev libzstd1 libzstd1-dev zstd
nothing else
end result in bionic:
ii libzstd-dev 1.3.3+dfsg-1ubuntu1
ii libzstd1 1.3.3+dfsg-1ubuntu1
ii libzstd1-dev 1.3.3+dfsg-1ubuntu1
ii zstd 1.3.3+dfsg-1ubuntu1
- without the transitional package installed:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
do-release-upgrade stats:
upgrade list: libzstd-dev libzstd1 zstd
nothing else
end result in bionic:
ii libzstd-dev 1.3.3+dfsg-1ubuntu1
ii libzstd1 1.3.3+dfsg-1ubuntu1
ii zstd 1.3.3+dfsg-1ubuntu1
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2018-02-05 19:26:20 |
Launchpad Janitor |
libzstd (Ubuntu Bionic): status |
In Progress |
Fix Released |
|
| 2018-02-15 11:31:07 |
Robie Basak |
libzstd (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2018-02-15 11:31:10 |
Robie Basak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-02-15 11:31:12 |
Robie Basak |
bug |
|
|
added subscriber SRU Verification |
| 2018-02-15 11:31:16 |
Robie Basak |
tags |
|
verification-needed verification-needed-xenial |
|
| 2018-02-15 11:32:01 |
Robie Basak |
libzstd (Ubuntu Artful): status |
In Progress |
Fix Committed |
|
| 2018-02-15 11:32:05 |
Robie Basak |
tags |
verification-needed verification-needed-xenial |
verification-needed verification-needed-artful verification-needed-xenial |
|
| 2018-02-15 13:29:27 |
Dan Streetman |
removed subscriber Dan Streetman |
|
|
|
| 2018-02-20 18:36:08 |
Andreas Hasenack |
description |
[Special SRU Notes]
The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path.
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
There are three major test cases: compression tests, package upgrade tests, and release upgrade tests.
This SRU has so many tests because it's introducing a new package and a carefully thought out upgrade plan.
a) compression tests
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Zesty:
$ zstd /usr/bin/snap -o snap.1.1.2.zst
/usr/bin/snap : 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Zesty:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Zesty:
$ zstd -d snap.1.1.2.zst -o snap.1.1.2
snap.1.1.2.zst : 10825608 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 10825608 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Zesty:
$ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1
ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2
ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Zesty:
$ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In zesty it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 9909192 bytes
* Still in zesty, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
7c980688861eef598a56c7970d952028 snap.1.3.1-new
7c980688861eef598a56c7970d952028 /usr/bin/snap
b) package upgrade tests
b.1)XENIAL
* Install the xenial original packages:
$ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1
* Enable xenial-proposed and verify that a dist-upgrade will upgrade zstd, pull in the new libzstd1, and leave libzstd-dev alone:
$ sudo apt dist-upgrade
...
The following NEW packages will be installed:
libzstd1
The following packages will be upgraded:
zstd
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
* In the end you will have these installed, at these versions:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
* Installing the new versioned dev package should remove the unversioned one:
$ sudo apt install libzstd1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
libzstd-dev
The following NEW packages will be installed:
libzstd1-dev
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
* The end result being:
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
* Conversely, installing the unversioned dev package removes the versioned one::
Installing libzstd-dev removes libzstd1-dev:
$ sudo apt install libzstd-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
libzstd1-dev
The following NEW packages will be installed:
libzstd-dev
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
* Start from scratch and reinstall the original xenial zstd packages:
$ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1
* Verify that a simple apt upgrade (not dist-upgrade) upgrades zstd and installs the new libzstd1:
$ sudo apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
libzstd1
The following packages will be upgraded:
zstd
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 392 kB of archives.
After this operation, 904 kB of additional disk space will be used.
Do you want to continue? [Y/n]
* After the upgrade, you should have:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
b.2) ARTFUL
* starting with all original artful zstd packages installed:
ii libzstd-dev 1.3.1+dfsg-1
ii libzstd1 1.3.1+dfsg-1
ii zstd 1.3.1+dfsg-1
* a dist-upgrade upgrades them just fine, with no new dependencies introduced:
The following packages will be upgraded:
libzstd-dev libzstd1 zstd
3 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
* which results in:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
* libzstd1-dev, the transitional package, can also be installed just fine:
$ sudo apt install libzstd1-dev
The following NEW packages will be installed:
libzstd1-dev
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
* removing libzstd-dev also removes libzstd1-dev:
The following packages will be REMOVED:
libzstd-dev* libzstd1-dev*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
* installing just the new libzstd1-dev pull in the unversioned dev package:
$ sudo apt install libzstd1-dev
The following additional packages will be installed:
libzstd-dev libzstd1
The following NEW packages will be installed:
libzstd-dev libzstd1 libzstd1-dev
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
c) release upgrade tests
RELEASE UPGRADE FROM XENIAL TO ARTFUL
1) starting with the 0.5.1 dev package in xenial:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii zstd 0.5.1-1
dist-upgrade to proposed version:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
in the do-release-upgrade upgrade list:
libzstd-dev libzstd1 zstd
deemed obsolete and removed in the end: libzstd0
end result in artful:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
2) starting with the 1.3.1 dev package in xenial:
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
do-release-upgrade stats:
install list: libzstd-dev
upgrade list: libzstd1 libzstd1-dev zstd
remove (auto installed): libzstd0
end result:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
RELEASE UPGRADE FROM ARTFUL TO BIONIC
- with all zstd packages installed:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
do-release-upgrade stats:
upgrade list: libzstd-dev libzstd1 libzstd1-dev zstd
nothing else
end result in bionic:
ii libzstd-dev 1.3.3+dfsg-1ubuntu1
ii libzstd1 1.3.3+dfsg-1ubuntu1
ii libzstd1-dev 1.3.3+dfsg-1ubuntu1
ii zstd 1.3.3+dfsg-1ubuntu1
- without the transitional package installed:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
do-release-upgrade stats:
upgrade list: libzstd-dev libzstd1 zstd
nothing else
end result in bionic:
ii libzstd-dev 1.3.3+dfsg-1ubuntu1
ii libzstd1 1.3.3+dfsg-1ubuntu1
ii zstd 1.3.3+dfsg-1ubuntu1
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
[Special SRU Notes]
The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path.
[Impact]
* libzstd v0.5.1 is an experimental version,
which generates and read an experimental format
incompatible with official libzstd v1+ format.
* Backporting a newer version >= v1.0, such as artful's v1.3.1,
fixes this issue
* This backport/SRU is being requested by upstream.
[Test Case]
There are three major test cases: compression tests, package upgrade tests, and release upgrade tests.
This SRU has so many tests because it's introducing a new package and a carefully thought out upgrade plan.
a) compression tests
* with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this:
Xenial:
$ zstd /usr/bin/snap -o snap.0.5.1.zst
Compressed 15528016 bytes into 4134889 bytes ==> 26.63%
Artful:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 31.03% (11318760 => 3512707 bytes, snap.1.3.1.zst)
* Upgrade to the package made available through this SRU and compress it again:
Xenial:
$ zstd /usr/bin/snap -o snap.1.3.1.zst
/usr/bin/snap : 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst)
Artful:
$ zstd /usr/bin/snap -o snap.1.3.1-updated.zst
/usr/bin/snap : 31.03% (11318760 => 3512707 bytes, snap.1.3.1-updated.zst)
* Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one:
Xenial:
$ zstd -d snap.0.5.1.zst -o snap.0.5.1
snap.0.5.1.zst : 15528016 bytes
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 15528016 bytes
Artful:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1
snap.1.3.1.zst : 11318760 bytes
$ zstd -d snap.1.3.1-updated.zst -o snap.1.3.1-updated
snap.1.3.1-updated.zst: 11318760 bytes
* Take md5sums of all uncompressed files and the original. They must of course match:
Xenial:
$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1
0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap
0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1
0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1
Artful:
$ md5sum /usr/bin/snap snap.1.3.1 snap.1.3.1-updated
0b65998bc775d444b2ee7d00cd18634a /usr/bin/snap
0b65998bc775d444b2ee7d00cd18634a snap.1.3.1
0b65998bc775d444b2ee7d00cd18634a snap.1.3.1-updated
* Downgrade zstd back to the original version of the ubuntu release you are testing:
Xenial:
$ sudo apt install zstd=0.5.1-1
Artful:
$ sudo apt install zstd=1.3.1+dfsg-1 libzstd1=1.3.1+dfsg-1
* Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format:
$ zstd -d snap.1.3.1.zst -o /dev/null
zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y
zstd: snap.1.3.1.zst: not in zstd format
In artful it should work:
$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new
snap.1.3.1.zst : 11318760 bytes
* Still in artful, compare the md5, which should match:
$ md5sum snap.1.3.1-new /usr/bin/snap
0b65998bc775d444b2ee7d00cd18634a snap.1.3.1-new
0b65998bc775d444b2ee7d00cd18634a /usr/bin/snap
b) package upgrade tests
b.1)XENIAL
* Install the xenial original packages:
$ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1
* Enable xenial-proposed and verify that a dist-upgrade will upgrade zstd, pull in the new libzstd1, and leave libzstd-dev alone:
$ sudo apt dist-upgrade
...
The following NEW packages will be installed:
libzstd1
The following packages will be upgraded:
zstd
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
* In the end you will have these installed, at these versions:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
* Installing the new versioned dev package should remove the unversioned one:
$ sudo apt install libzstd1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
libzstd-dev
The following NEW packages will be installed:
libzstd1-dev
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
* The end result being:
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
* Conversely, installing the unversioned dev package removes the versioned one::
Installing libzstd-dev removes libzstd1-dev:
$ sudo apt install libzstd-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
libzstd1-dev
The following NEW packages will be installed:
libzstd-dev
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
* Start from scratch and reinstall the original xenial zstd packages:
$ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1
* Verify that a simple apt upgrade (not dist-upgrade) upgrades zstd and installs the new libzstd1:
$ sudo apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
libzstd1
The following packages will be upgraded:
zstd
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 392 kB of archives.
After this operation, 904 kB of additional disk space will be used.
Do you want to continue? [Y/n]
* After the upgrade, you should have:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
b.2) ARTFUL
* starting with all original artful zstd packages installed:
ii libzstd-dev 1.3.1+dfsg-1
ii libzstd1 1.3.1+dfsg-1
ii zstd 1.3.1+dfsg-1
* a dist-upgrade upgrades them just fine, with no new dependencies introduced:
The following packages will be upgraded:
libzstd-dev libzstd1 zstd
3 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
* which results in:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
* libzstd1-dev, the transitional package, can also be installed just fine:
$ sudo apt install libzstd1-dev
The following NEW packages will be installed:
libzstd1-dev
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
* removing libzstd-dev also removes libzstd1-dev:
The following packages will be REMOVED:
libzstd-dev* libzstd1-dev*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
* installing just the new libzstd1-dev pull in the unversioned dev package:
$ sudo apt install libzstd1-dev
The following additional packages will be installed:
libzstd-dev libzstd1
The following NEW packages will be installed:
libzstd-dev libzstd1 libzstd1-dev
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
c) release upgrade tests
RELEASE UPGRADE FROM XENIAL TO ARTFUL
1) starting with the 0.5.1 dev package in xenial:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii zstd 0.5.1-1
dist-upgrade to proposed version:
ii libzstd-dev 0.5.1-1
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
in the do-release-upgrade upgrade list:
libzstd-dev libzstd1 zstd
deemed obsolete and removed in the end: libzstd0
end result in artful:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
2) starting with the 1.3.1 dev package in xenial:
ii libzstd0 0.5.1-1
ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1
ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1
ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1
do-release-upgrade stats:
install list: libzstd-dev
upgrade list: libzstd1 libzstd1-dev zstd
remove (auto installed): libzstd0
end result:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
RELEASE UPGRADE FROM ARTFUL TO BIONIC
- with all zstd packages installed:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
do-release-upgrade stats:
upgrade list: libzstd-dev libzstd1 libzstd1-dev zstd
nothing else
end result in bionic:
ii libzstd-dev 1.3.3+dfsg-1ubuntu1
ii libzstd1 1.3.3+dfsg-1ubuntu1
ii libzstd1-dev 1.3.3+dfsg-1ubuntu1
ii zstd 1.3.3+dfsg-1ubuntu1
- without the transitional package installed:
ii libzstd-dev 1.3.1+dfsg-1ubuntu0.1
ii libzstd1 1.3.1+dfsg-1ubuntu0.1
ii zstd 1.3.1+dfsg-1ubuntu0.1
do-release-upgrade stats:
upgrade list: libzstd-dev libzstd1 zstd
nothing else
end result in bionic:
ii libzstd-dev 1.3.3+dfsg-1ubuntu1
ii libzstd1 1.3.3+dfsg-1ubuntu1
ii zstd 1.3.3+dfsg-1ubuntu1
[Regression Potential]
* During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update.
* Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1.
[Other Info]
* Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data.
* On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression.
--- Original description ---
Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial.
Reason for the backport:
========================
Current version in Xenial is v0.5.1,
it's an experimental version, using its own, incompatible format.
As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world.
This is of important concern for products using libzstd as a shared library :
on Ubuntu Xenial, produced data is different, not compatible with v1+.
This issue has been made more pressing with the integration of zstd in Linux Kernel,
as userland tools must also be updated to read and generate zstd.
Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037
Testing:
========
Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing.
Backport available from PPA:
https://launchpad.net/~ginggs/+archive/ubuntu/backports
* xenial:
[X] Package builds without modification (needs debhelper 10)
[X] zstd installs cleanly and runs
[ ] zstd-dbgsym installs cleanly and runs
[ ] libzstd1-dbgsym installs cleanly and runs
[X] libzstd-dev installs cleanly and runs
[X] libzstd1 installs cleanly and runs
No reverse dependencies |
|
| 2018-02-20 20:09:42 |
Andreas Hasenack |
tags |
verification-needed verification-needed-artful verification-needed-xenial |
verification-done-artful verification-done-xenial verification-needed |
|
| 2018-02-22 21:12:08 |
Frank Holton |
bug |
|
|
added subscriber Frank Holton |
| 2018-03-01 10:03:03 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2018-03-01 10:13:06 |
Launchpad Janitor |
libzstd (Ubuntu Artful): status |
Fix Committed |
Fix Released |
|
| 2018-03-05 09:08:22 |
Launchpad Janitor |
libzstd (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
