Please compress packages with zstd by default

[Balint Reczey]

https://people.canonical.com/~rbalint/zstd-debs/ contains a .deb built on Hirsute having both data and control members of the .deb being compressed with zstd. It can be handy for testing various tools.

[dpkg]
Decompression support in dpkg landed first in Bionic and is being SRUd to Xenial in LP: #1764220 enable Launchpad's Xenial systems to process the zstd-compressed binary packages.
From dpkg's perspective the upgrade path is cleared.

The original plan was compressing only the internal data.tar .deb member, but dpkg uses uniform compression by default since dpkg 1.19.0 thus I'm collecting all the changes to support control.tar.zst, too, in this bug.

Reviewed packages from:
https://codesearch.debian.net/search?q=data.tar.xz&literal=1&perpkg=1
https://codesearch.debian.net/search?q=control.tar.xz&literal=1&perpkg=1

appstream-glib - needs fix: libappstream-builder/asb-package-deb.c
aptly - needs fix: deb/deb.go
boinc - needs fix: debian/fetch_example_applications.sh
busybox - needs fix: archival/dpkg_deb.c archival/dpkg.c
cdebootstrap - needs fix: src/package.c
cdist - may need fix, can use dpkg-deb: cdist/preos/debootstrap/files/devuan-debootstrap/functions
debdelta - needs fix: debdelta debpatch.sh
debian-el - needs fix: deb-view.el
debian-handbook - needs fix, maybe later, for Debian
debootstrap - needs fix, https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/54
debsigs - needs fix, debsigs
debsig-verify - needs fix, src/debsig-verify.c
diffoscope - needs fix, diffoscope/comparators/deb.py
dpkg - needs fix, change default
dpkg-sig - needs fix, dpkg-sig
dpmb - needs fix, maybe later, for Debian
elfutils - may need fix, uses dpkg-deb if it is available, does not handle .gz either
file - needs fix, magic/Magdir/archive
libsolv - needs fix, ext/repo_deb.c
lintian - needs fix malformed-deb-archive
lutris - needs fix, lutris/util/extract.py
obs-build - needs fix Build/Deb.pm
osc - needs fix osc/util/debquery.py control.tar.zst only
python-apt - needs fix apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall()
radare2 - needs fix
reprepro - needs fix, debfile.c
vim-scripts - needs fix debPlugin/autoload/deb.vim
winetricks - needs fix when Debian switches src/winetricks
zeroinstall-injector - needs fix src/zeroinstall/archive.ml

acr - skip, does not _have to_ be fixed, just creates packages, see dist/deb_hand.mak
alien - skip, uses dpkg-deb to extract .deb
ansible - not affected, just test data in dbdata.tar.xz
anthy - not affected, just changelog entry
apt - seems fixed already
ceph - not affected in Ubuntu's version
circlator - not affected, just test data
cowdancer - not affected, just documentation
eccodes - skip, just orig-data.tar.xz
eckit - skip, just ...orig-data.tar.xz
firefox - skip, profdata.tar.xz
firefox-esr - skip, profdata.tar.xz
galculator - skip, just changelog
grads - skip, ...orig-data.tar.xz
gvmd - skip, just creates xz compressed .deb
insighttoolkit4 - skip, ...orig-data.tar.xz
jdeb - skip, just creates compressed .deb packages
jmol - skip, just local data archive
jq - skip, extracting .deb in Tracis CI only
libldm - skip, just local data archive
linux - skip, just local data archive
mate-common - skip, just changelog
megaglest - skip, just local data archive
metview - skip, ...orig-data.tar.xz
mir - skip, .gitignore
multiqc - skip, false hit
naev - skip, local data tarball
nanofilt - skip, local data tarball
python-dendropy - skip, just local data archive
python-gear - skip, just changelog
python-x2go - skip, just changelog
salt - skip, extracts only a fixture
streamlink - skip, changelog only
thunderbird - skip, local data tarballs
velvet - skip, changelog only
veyon - skip, just changelog
victoriametrics - skip, just assembles .deb
wlcs - skip, Travis CI only
wpa - skip, changelog only

[Dimitri John Ledkov (xnox)]

# file perl-base_5.32.1-3ubuntu2_amd64.deb
perl-base_5.32.1-3ubuntu2_amd64.deb: Debian binary package (format 2.0), with control.tar.zs, data compression zst

Looks odd

[Dimitri John Ledkov (xnox)]

# display 3rd character of file name extension like 2 of bz2 or m of lzma
>>>>>&-1 ubyte x \b%c
>>>>>>&0 ubyte !0x20
>>>>>>>&-1 ubyte !0x2f

is odd.

[Balint Reczey (rbalint)]

Debootstrap fix is merged in Debian:
https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/54

[Julian Andres Klode (juliank)]

I don't understand what python-apt does here, it inherits zstd support from apt.

[Julian Andres Klode (juliank)]

There seems to be a bug in bionic's apt.

root@bb:/home/jak/Projects# python3 -c 'import apt_inst, apt_pkg; apt_pkg.init(); apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2_zstd1_all.deb").control.extractdata("control")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
LookupError: There is no member named 'control'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 25, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 35, in <module>
    apt_pkg.init_system()
apt_pkg.Error: E:Cannot find a configured compressor for 'false'

[Julian Andres Klode (juliank)]

I have not tested hirsute in a clean container; but touch /usr/bin/zstd makes it work, so the compressor lookup is broken.

[Balint Reczey (rbalint)]

@juliank Yes, python-apt works with just touched /usr/bin/zstd :

>>> apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall()
True

[Balint Reczey (rbalint)]

@juliank Xenial's apt/python3-apt does not work even with zstd installed:

root@x-zstd:~# python3
Python 3.5.2 (default, Jan 26 2021, 13:30:48)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import apt_inst, apt_pkg
>>> apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: E:Internal error, could not locate member control.tar.{lz4gzxzbz2lzma}
>>>
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================-=======================-=======================-================================================================================
ii zstd 1.3.1+dfsg-1~ubuntu0.16 amd64 fast lossless compression algorithm -- CLI tool

[Julian Andres Klode (juliank)]

Aside from the code being wrong - it's missing an apt_pkg.init_config(), it also does not work with the config being setup; it needs to be whitelisted.

So we definitely need to SRU APT. However, we are already preparing 1.2.35 at the moment for ESM enablement, I don't know if we want to stuff both things together.

[Balint Reczey (rbalint)]

@juliank IMO it would be better to add the zstd-related changes, too, to the planned SRU if those are not risky and the delay due to adding them is acceptable for ESM's perspective.

[Julian Andres Klode (juliank)]

Split out APT / python-apt into bug 1923845

[Launchpad Janitor (janitor)]

This bug was fixed in the package debootstrap - 1.0.124

---------------
debootstrap (1.0.124) experimental; urgency=medium

  [ Colin Watson ]
  * Add (Ubuntu) groovy as a symlink to gutsy.

  [ Samuel Thibault ]
  * functions: Turn for loops variables into locals, suggested by jhcha54008.

  [ YunQiang Su ]
  * stage1: re-mkdir /proc if it is a symlink instead of umount. (Closes: #968927)

  [ Luca Falavigna ]
  * Add (Ubuntu) hirsute as a symlink to gutsy.

  [ Tobias Koch ]
  * Add --inrelease-path option to allow specifying by-hash InRelease
    location.

  [ Dimitri John Ledkov ]
  * functions: add zstd control&data support to the ar extractor LP: #1923845
  * Add (Ubuntu) impish as a symlink to gutsy.
  * gutsy: sync changes from Ubuntu, to ensure that impish+ are always
    merged-usr.
  * gutsy: use --extractor=ar if dpkg-deb has no zstd support (i.e. on
    Debian & non-Ubuntu derivatives).
  * rules,control: adjust Recommends/Suggests to be appropriate on each
    distro.

 -- Dimitri John Ledkov <email address hidden> Mon, 26 Apr 2021 15:02:44 +0100

[Balint Reczey (rbalint)]

Verified 1.6.5ubuntu0.6 on Bionic.

root@bb-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:29:17-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’

glibc-doc-reference_2.33-0ubuntu2~zstd1_all 100%[=========================================================================================>] 5.36M 4.80MB/s in 1.1s

2021-05-13 19:29:18 (4.80 MB/s) - ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]

root@bb-lp1926437:~# python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import apt_inst
>>> import apt_pkg
>>> apt_pkg.init_config()
>>> apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall()
True
>>>
root@bb-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================-=======================-=======================-================================================================================
ii python-apt-common 1.6.5ubuntu0.6 all Python interface to libapt-pkg (locales)
ii python3-apt 1.6.5ubuntu0.6 amd64 Python 3 interface to libapt-pkg
root@bb-lp1926437:~#

[Balint Reczey (rbalint)]

Verified 2.2.0~ubuntu0.21.04.1 on Hirsute.

root@hh-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:42:59-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’

glibc-doc-reference_2.33-0ubuntu2~zstd1_all 100%[=========================================================================================>] 5.36M 4.78MB/s in 1.1s

2021-05-13 19:43:00 (4.78 MB/s) - ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]

root@hh-lp1926437:~# python3 -c 'import apt_inst; import apt_pkg; apt_pkg.init_config(); print(apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall())'
True
root@hh-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-=====================-============-========================================
ii python-apt-common 2.2.0~ubuntu0.21.04.1 all Python interface to libapt-pkg (locales)
ii python3-apt 2.2.0~ubuntu0.21.04.1 amd64 Python 3 interface to libapt-pkg

[Balint Reczey (rbalint)]

Verified 2.1.3ubuntu1.4 on Groovy.

root@gg-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:51:34-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’

glibc-doc-reference_2.33-0ubuntu2~zstd1_all 100%[=========================================================================================>] 5.36M 4.70MB/s in 1.1s

2021-05-13 19:51:36 (4.70 MB/s) - ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]

root@gg-lp1926437:~# python3 -c 'import apt_inst; import apt_pkg; apt_pkg.init_config(); print(apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall())'
True
root@gg-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-==============-============-========================================
ii python-apt-common 2.1.3ubuntu1.4 all Python interface to libapt-pkg (locales)
ii python3-apt 2.1.3ubuntu1.4 amd64 Python 3 interface to libapt-pkg

[Balint Reczey (rbalint)]

Verified 2.0.0ubuntu0.20.04.5 on Focal.

root@f-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:56:02-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’

glibc-doc-reference_2.33-0ubuntu2~zstd1_all 100%[=========================================================================================>] 5.36M 4.61MB/s in 1.2s

2021-05-13 19:56:03 (4.61 MB/s) - ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]

root@f-lp1926437:~# python3 -c 'import apt_inst; import apt_pkg; apt_pkg.init_config(); print(apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall())'
True
root@f-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-====================-============-========================================
ii python-apt-common 2.0.0ubuntu0.20.04.5 all Python interface to libapt-pkg (locales)
ii python3-apt 2.0.0ubuntu0.20.04.5 amd64 Python 3 interface to libapt-pkg

[Balint Reczey (rbalint)]

Sorry, those comments were for LP: #1926437, which is at least related. :-)

[Balint Reczey (rbalint)]

The attached dpkg patch has been tested in the following Bileto ticket:

https://bileto.ubuntu.com/#/ticket/4583

Lintian needs to be fixed first to ensure smooth migration. The software-properties failure does not seem to be related at first glance.

[Dimitri John Ledkov (xnox)]

dpkg patch looks good to me.

did lintian patch locally and started to run the regression test suite locally to see if anything else is tripped up.

working on reprepro changes too.

[Dimitri John Ledkov (xnox)]

lintian should be good in impish-proposed, but we should migrate it first.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lintian - 2.104.0ubuntu3

---------------
lintian (2.104.0ubuntu3) impish; urgency=medium

  * Add zst as allowed compression for .deb & .udeb. Although we don't
    actually ship udeb package that supports zst. LP: #1923845

 -- Dimitri John Ledkov <email address hidden> Tue, 15 Jun 2021 10:42:12 +0100

[Balint Reczey (rbalint)]

@xnox Thanks! Please also SRU the reprepro changes. Ideally users of reprepro on stable releases would be able to just upgrade reprepro from -updates before the first zstd-compressed .deb lands in Impish to not break their setup even when they maintain a local repository of the devel series.

[Launchpad Janitor (janitor)]

This bug was fixed in the package hello - 2.10-2ubuntu3

---------------
hello (2.10-2ubuntu3) impish; urgency=medium

  * Ubuntu Impish will be switching default dpkg compression format to
    Zstd. This upload is the first canary to signal the archive format
    change. dpkg in Ubuntu supports zstd in xenial and up. If any issues
    are identified with this package, please report them at
    https://pad.lv/1923845 LP: #1923845

 -- Dimitri John Ledkov <email address hidden> Fri, 18 Jun 2021 11:58:54 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package reprepro - 5.3.0-1.3

---------------
reprepro (5.3.0-1.3) experimental; urgency=medium

  * Non-maintainer upload.
  * Add support for Zstd compressed debs. (LP: #1923845)

 -- Dimitri John Ledkov <email address hidden> Mon, 21 Jun 2021 10:16:53 +0100

[Balint Reczey (rbalint)]

Proposed diffoscope fix: https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/83

[Balint Reczey (rbalint)]

Diffoscope already handles zstd compressed members well.

[Brian Murray (brian-murray)]

Hello Balint, or anyone else affected,

Accepted reprepro into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/reprepro/5.3.0-1.3~ubuntu21.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Balint, or anyone else affected,

Accepted reprepro into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/reprepro/5.3.0-1.3~ubuntu20.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Balint, or anyone else affected,

Accepted reprepro into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/reprepro/5.3.0-1.3~ubuntu20.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted reprepro (5.3.0-1.3~ubuntu20.04) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

dgit/9.10 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#reprepro

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Dimitri John Ledkov (xnox)]

See reprepro verification comments at https://bugs.launchpad.net/ubuntu/+source/reprepro/+bug/1933363

[Dimitri John Ledkov (xnox)]

@ubuntu-sru-bot

regressions triggered by reprepro have been retried, and have been now cleared.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dpkg - 1.20.9ubuntu2

---------------
dpkg (1.20.9ubuntu2) impish; urgency=medium

  * build: Switch default dpkg-deb compression from xz to zstd. (LP: #1923845)
    Keep compressing dpkg.deb with xz to help bootstrapping on non-Ubuntu
    systems.

 -- Balint Reczey <email address hidden> Mon, 14 Jun 2021 16:04:10 +0200

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for reprepro has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package reprepro - 5.3.0-1.3~ubuntu20.04

---------------
reprepro (5.3.0-1.3~ubuntu20.04) focal; urgency=medium

  * Backport Zstd compressed debs support to hirsute. LP: #1933363

reprepro (5.3.0-1.3) experimental; urgency=medium

  * Non-maintainer upload.
  * Add support for Zstd compressed debs. (LP: #1923845)

reprepro (5.3.0-1.2) unstable; urgency=medium

  * Non-maintainer upload.
  * Flush stdout and stderr before execv of an end hook (Closes: #982423)

 -- Dimitri John Ledkov <email address hidden> Wed, 23 Jun 2021 17:01:44 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package reprepro - 5.3.0-1.3~ubuntu20.10

---------------
reprepro (5.3.0-1.3~ubuntu20.10) groovy; urgency=medium

  * Backport Zstd compressed debs support to hirsute. LP: #1933363

reprepro (5.3.0-1.3) experimental; urgency=medium

  * Non-maintainer upload.
  * Add support for Zstd compressed debs. (LP: #1923845)

reprepro (5.3.0-1.2) unstable; urgency=medium

  * Non-maintainer upload.
  * Flush stdout and stderr before execv of an end hook (Closes: #982423)

 -- Dimitri John Ledkov <email address hidden> Wed, 23 Jun 2021 17:01:44 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package reprepro - 5.3.0-1.3~ubuntu21.04

---------------
reprepro (5.3.0-1.3~ubuntu21.04) hirsute; urgency=medium

  * Backport Zstd compressed debs support to hirsute. LP: #1933363

reprepro (5.3.0-1.3) experimental; urgency=medium

  * Non-maintainer upload.
  * Add support for Zstd compressed debs. (LP: #1923845)

 -- Dimitri John Ledkov <email address hidden> Wed, 23 Jun 2021 17:01:44 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package diffoscope - 179

---------------
diffoscope (179) unstable; urgency=medium

  * Ensure that various LLVM tools are installed, even when testing whether
    a MacOS binary has zero differences when compared to itself.
    (Closes: reproducible-builds/diffoscope#270)

 -- Chris Lamb <email address hidden> Fri, 30 Jul 2021 09:05:01 +0100

[Athos Ribeiro (athos-ribeiro)]

dh-cmake currently FTBFS [1].

While investigating the issue, I realized that, now dpkg-deb defaults to compressing with zstd, python-debian can no longer decompress the compressed data into the binary package archive.

I am proposing a patch for python-debian in the following MP: https://code.launchpad.net/~athos-ribeiro/ubuntu/+source/python-debian/+git/python-debian/+merge/407413

A PPA with the proposed fix is available at https://launchpad.net/~athos-ribeiro/+archive/ubuntu/lp-1923845-python-debian/+packages

[1] https://launchpadlibrarian.net/552708462/buildlog_ubuntu-impish-amd64.dh-cmake_0.6.1_BUILDING.txt.gz

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-debian - 0.1.39ubuntu1

---------------
python-debian (0.1.39ubuntu1) impish; urgency=medium

  * Add support for zstd compression format (LP: #1923845)

 -- Athos Ribeiro <email address hidden> Thu, 19 Aug 2021 14:50:49 -0300

[Stephan Lachnit (stephanlachnit)]

FYI I added initial support for Lutris here (https://github.com/lutris/lutris/pull/4021).
However this is really only cosmetics, it won't work until an stdlib zstd module is available in Python (see https://<email address hidden>/thread/VQIFA7WTNRAOYZGTVP4WZC2CD36KYIVY/#RMMZFYPKHDJHGSHCKHRKRDGKPZSCUHT2).

[Paride Legovini (paride)]

aptly is Fix Release in Kinetic in:

aptly (1.4.0+ds1-7) unstable; urgency=medium

  * Team upload.
  * Add support for zstd compression (Closes: #1010465)

[Luca Boccassi (bluca)]

SUSE and Fedora have started to compress repodata with zstd, so it is now impossible to bootstrap those distributions from Ubuntu via dnf/zypper due to the lack of zstd support in libsolve. This affects my CI jobs running in Jammy.
The fix is trivial as the version already supports it, but it needs to build-depend on libzstd and enable the build config flag for it. I will prepare SRUs for mantic/lunar/jammy. It will be fixed in noble with the next sync from Debian unstable, at version 0.7.26-2.

[Jay Fulton (databasejay)]

Thank you! Keepin' it alive.

On Fri, Nov 17, 2023 at 6:36 AM Luca Boccassi <email address hidden>
wrote:

> ** Changed in: libsolv (Ubuntu)
> Status: Confirmed => Fix Released
>
> --
> You received this bug notification because you are a member of Ubuntu
> Massachusetts LoCo, which is subscribed to hello in Ubuntu.
> https://bugs.launchpad.net/bugs/1923845
>
> Title:
> Please compress packages with zstd by default
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/appstream-glib/+bug/1923845/+subscriptions
>
>
> --
> Ubuntu-us-ma mailing list
> <email address hidden>
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-ma
>