xz compressed control.tar files not supported

Bug #1730627 reported by Witold Szczeponik on 2017-11-07
74
This bug affects 14 people
Affects Status Importance Assigned to Milestone
dpkg (Ubuntu)
Undecided
Unassigned
Precise
Medium
Unassigned
Trusty
Medium
Adam Conrad

Bug Description

[impact]

trusty dpkg fails when trying to extract a dpkg source package compressed using xz. This breaks running autopkgtest from bionic with any and all trusty packages.

[test case]

on a bionic system, get any trusty source dpkg and run autopkgtest with it. See comment 11 for an example.

[regression potential]

any changes to dpkg has the potential to break all package changes on the system, which obviously is a serious regression. However, this specific change has been upstream for a very long time. Any breakage would happen when dpkg is either building or extracting a deb.

[other info]

this affects debootstrap as well, as described in the original bug description:

---

trusty's dpkg (1.17.5ubuntu5.7) cannot handle .deb files that contain control.tar files that are compressed using the xz compressor.

Here's an example of such a case when trying to show the contents of a .deb file from bionic's repositories:

dpkg-deb: error: archive './var/cache/apt/archives/ubuntu-minimal_1.406_amd64.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up

This issue makes it currently impossible to create a bionic environment using debootstrap (currently from trusty-proposed).

I have observed the issue on trusty, precisely on

$ lsb_release -rd
Description: Ubuntu 14.04.5 LTS
Release: 14.04

using

$ apt-cache policy dpkg
dpkg:
  Installed: 1.17.5ubuntu5.7
  Candidate: 1.17.5ubuntu5.7
  Version table:
 *** 1.17.5ubuntu5.7 0
        500 mirror://mirrors.ubuntu.com/mirrors.txt/ trusty-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.17.5ubuntu5.6 0
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
     1.17.5ubuntu5 0
        500 mirror://mirrors.ubuntu.com/mirrors.txt/ trusty/main amd64 Packages

Witold Szczeponik (wsz) wrote :

Commit c17be3cbfc58e5b54ae1d5ae4714460d7ae2e15c from git://anonscm.debian.org/dpkg/dpkg.git seems to contain a fix.

It can be applied to 1.17.5ubuntu5.7, and I successfully built a package. With said patch applied, xz compressed control.tar files seem to be supported.

Witold Szczeponik (wsz) on 2017-11-07
tags: added: trusty
Brian Murray (brian-murray) wrote :
Changed in dpkg (Ubuntu):
status: New → Fix Released
Changed in dpkg (Ubuntu Precise):
importance: Undecided → Medium
Changed in dpkg (Ubuntu Trusty):
importance: Undecided → Medium
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in dpkg (Ubuntu Precise):
status: New → Confirmed
Changed in dpkg (Ubuntu Trusty):
status: New → Confirmed
Anthony Yarbrough (virento) wrote :
Download full text (3.2 KiB)

I'm experiencing this issue when attempting to install the latest Liquorix kernel that I've been using for years.

$ lsb_release -rd
Description: Ubuntu 14.04.5 LTS
Release: 14.04

$ apt-cache policy dpkg
dpkg:
  Installed: 1.17.5ubuntu5.7
  Candidate: 1.17.5ubuntu5.7
  Version table:
 *** 1.17.5ubuntu5.7 0
        500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.17.5ubuntu5.6 0
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
     1.17.5ubuntu5 0

$ sudo apt full-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  linux-headers-4.13.0-13.1-liquorix-amd64
  linux-image-4.13.0-13.1-liquorix-amd64
The following packages will be upgraded:
  linux-headers-liquorix-amd64 linux-image-liquorix-amd64
2 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/57.4 MB of archives.
After this operation, 280 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Supported
Create a snapshot of '/tmp/apt-btrfs-snapshot-mp-21ujpnev/@' in '/tmp/apt-btrfs-snapshot-mp-21ujpnev/@apt-snapshot-2017-11-19_17:41:21'
Preconfiguring packages ...
dpkg-deb: error: archive '/var/cache/apt/archives/linux-image-4.13.0-13.1-liquorix-amd64_4.13-7_amd64.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up
dpkg: error processing archive /var/cache/apt/archives/linux-image-4.13.0-13.1-liquorix-amd64_4.13-7_amd64.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 2
dpkg-deb: error: archive '/var/cache/apt/archives/linux-headers-4.13.0-13.1-liquorix-amd64_4.13-7_amd64.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up
dpkg: error processing archive /var/cache/apt/archives/linux-headers-4.13.0-13.1-liquorix-amd64_4.13-7_amd64.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 2
dpkg-deb: error: archive '/var/cache/apt/archives/linux-headers-liquorix-amd64_4.13-7_amd64.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up
dpkg: error processing archive /var/cache/apt/archives/linux-headers-liquorix-amd64_4.13-7_amd64.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 2
dpkg-deb: error: archive '/var/cache/apt/archives/linux-image-liquorix-amd64_4.13-7_amd64.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up
dpkg: error processing archive /var/cache/apt/archives/linux-image-liquorix-amd64_4.13-7_amd64.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 2
No apport report written because MaxReports is reached already
                                                              Errors were encountered while processing:
 /var/cache/apt/archives/linux-image-4.13.0-13.1-liquorix-amd64_4.13-7_amd64.deb
 /var/cache/apt/archives/linux-headers-4.13.0-13.1-liquorix-amd64_4.13-7_amd64.deb
 /var/cache/apt/archives/linux-headers-liquorix-amd64_4.13-7_amd64.deb
 /var/cache/apt/archives/linux-image-liquorix-amd64_4.13-7_amd64.deb
[ Rootkit Hu...

Read more...

Adam Conrad (adconrad) on 2017-12-04
Changed in dpkg (Ubuntu Precise):
status: Confirmed → Won't Fix
Anthony Yarbrough (virento) wrote :

I can confirm that the method described in comment #1 by 'wsz' works like a charm for my Trusty installation and is much appreciated as I can now install the latest Liquorix kernel. Woohoo!

Adam Conrad (adconrad) on 2017-12-06
Changed in dpkg (Ubuntu Trusty):
assignee: nobody → Adam Conrad (adconrad)
Robie Basak (racb) wrote :

13:30 <rbasak> infinity: missing SRU paperwork for bug 1730627, but also, do we really need to SRU this? Is being able to debootstrap Bionic from Trusty important?
13:30 <ubottu> bug 1730627 in dpkg (Ubuntu Trusty) "xz compressed control.tar files not supported" [Medium,Confirmed] https://launchpad.net/bugs/1730627
13:31 <rbasak> We don't support an upgrade from Trusty to Bionic directly, so why do we need to support a debootstrap?

Witold Szczeponik (wsz) wrote :

I am running an environment (currently on Trusty) which supports building packages for the current Ubuntu and Debian releases for a non-trivial user group. For various reasons, I cannot update to Xenial in a timely manner, and since Bionic is looming large, I want (and need) to prepare for that release in a timely manner.

Hence my requirement for a "debootstrap" to Bionic on Trusty. Since a "debootstrap" fails for problems with "dpkg", I created this bug report.

Witold Szczeponik (wsz) wrote :

Adding to my comment #8: "debootstrap" is not only used for release upgrades, if I am not mistaken. Because if this was the main/only use case, the support for various vanilla Debian releases should not be included in "debootstrap" either.

Ross Gammon (rosco2) wrote :

Just adding a comment that this is about more than just bootstrapping.

Users on the Trusty LTS cannot install deb packages from upstream websites that were created with xz compression (ie. built with a newer distribution). This is how I came to this bug today.

Dan Streetman (ddstreet) wrote :

This bug breaks autopkgtest running on bionic for (apparently) all trusty packages.

For example:

$ lsb_release -r
Release: 18.04
$ pull-lp-source bash trusty
...
$ autopkgtest -U bash_4.3-7ubuntu1.7.dsc -- lxd ubuntu-daily:trusty
...
autopkgtest [13:31:50]: ERROR: "dpkg --unpack /tmp/autopkgtest.nywTmz/autopkgtest-satdep.deb" failed with stderr "dpkg-deb: error: archive '/tmp/autopkgtest.nywTmz/autopkgtest-satdep.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up

it's like that for all the trusty packages I tested.

Changed in dpkg (Ubuntu Trusty):
assignee: Adam Conrad (adconrad) → Dan Streetman (ddstreet)
Dan Streetman (ddstreet) wrote :

@infinity I'll get this patch SRUed into trusty.

Dan Streetman (ddstreet) wrote :

Built a test package with the upstream patch here:
https://launchpad.net/~ddstreet/+archive/ubuntu/lp1730627

Then tested autopkgtest on bionic system, with the patched dpkg. Fails as expected with the current trusty dpkg:

$ autopkgtest -U dpkg_1.17.5ubuntu5.8.dsc -- lxd ubuntu-daily:trusty
...
autopkgtest [15:20:15]: ERROR: "dpkg --unpack /tmp/autopkgtest.wuLHN3/autopkgtest-satdep.deb" failed with stderr "dpkg-deb: error: archive '/tmp/autopkgtest.wuLHN3/autopkgtest-satdep.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up

Build/autopkgtest successful with the patched trusty dpkg from PPA:

$ autopkgtest --setup-commands='sudo apt-add-repository ppa:ddstreet/lp1730627' -U dpkg_1.17.5ubuntu5.8.dsc -- lxd ubuntu-daily:trusty
...
dpkg-deb: building package `dpkg-dev' in `../dpkg-dev_1.17.5ubuntu5.8_all.deb'.
 dpkg-genchanges -b >../dpkg_1.17.5ubuntu5.8_amd64.changes
dpkg-genchanges: binary-only upload - not including any source code
 dpkg-source --after-build src
dpkg-source: info: using options from src/debian/source/options: --compression=xz
dpkg-buildpackage: binary only upload (no source included)
dpkg-source: info: using options from src/debian/source/options: --compression=xz
* SKIP no tests in this package
autopkgtest [15:29:22]: @@@@@@@@@@@@@@@@@@@@ summary
* SKIP no tests in this package

Dan Streetman (ddstreet) on 2018-02-23
description: updated
Adam Conrad (adconrad) on 2018-02-23
Changed in dpkg (Ubuntu Trusty):
assignee: Dan Streetman (ddstreet) → Adam Conrad (adconrad)

Hello Witold, or anyone else affected,

Accepted dpkg into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dpkg/1.17.5ubuntu5.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in dpkg (Ubuntu Trusty):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-trusty
Dan Streetman (ddstreet) wrote :

verification that it fixes the problem for me, re: autopkgtest. As that was the same cause/problem as initial reporter's, I expect it should be fixed for them as well.

ddstreet@thorin:~/bugs/lp1730627/verification$ pull-lp-source lsb-base trusty
pull-lp-source: Using source package 'lsb' for binary package 'lsb-base'
pull-lp-source: Found lsb 4.1+Debian11ubuntu6.2 in trusty
pull-lp-source: Good signature by Steve Langasek <email address hidden> (0x568D30F321B2133D)
pull-lp-source: Downloading lsb_4.1+Debian11ubuntu6.2.tar.xz from archive.ubuntu.com (0.063 MiB)
dpkg-source: info: extracting lsb in lsb-4.1+Debian11ubuntu6.2
dpkg-source: info: unpacking lsb_4.1+Debian11ubuntu6.2.tar.xz
ddstreet@thorin:~/bugs/lp1730627/verification$ autopkgtest -U lsb_4.1+Debian11ubuntu6.2.dsc -- lxd ubuntu-daily:trusty
autopkgtest [14:16:23]: version 5.1
autopkgtest [14:16:23]: host thorin.lan; command line: /usr/bin/autopkgtest -U lsb_4.1+Debian11ubuntu6.2.dsc -- lxd ubuntu-daily:trusty
...
(normal setup output)
...
autopkgtest [14:17:36]: testbed dpkg architecture: amd64
autopkgtest [14:17:38]: testbed running kernel: Linux 4.15.0-10-generic #11-Ubuntu SMP Tue Feb 13 18:23:35 UTC 2018
autopkgtest [14:17:38]: @@@@@@@@@@@@@@@@@@@@ source lsb_4.1+Debian11ubuntu6.2.dsc
autopkgtest [14:17:42]: ERROR: "dpkg --unpack /tmp/autopkgtest.Mrh8tN/autopkgtest-satdep.deb" failed with stderr "dpkg-deb: error: archive '/tmp/autopkgtest.Mrh8tN/autopkgtest-satdep.deb' has premature member 'control.tar.xz' before 'control.tar.gz', giving up
dpkg: error processing archive /tmp/autopkgtest.Mrh8tN/autopkgtest-satdep.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 2
Errors were encountered while processing:
 /tmp/autopkgtest.Mrh8tN/autopkgtest-satdep.deb

using -proposed package:

ddstreet@thorin:~/bugs/lp1730627/verification$ autopkgtest --apt-pocket proposed=dpkg -U lsb_4.1+Debian11ubuntu6.2.dsc -- lxd ubuntu-daily:trusty
autopkgtest [14:19:13]: version 5.1
autopkgtest [14:19:13]: host thorin.lan; command line: /usr/bin/autopkgtest --apt-pocket proposed=dpkg -U lsb_4.1+Debian11ubuntu6.2.dsc -- lxd ubuntu-daily:trusty
...
(normal setup output)
...
Preparing to unpack .../dpkg-dev_1.17.5ubuntu5.8_all.deb ...
Unpacking dpkg-dev (1.17.5ubuntu5.8) ...
...
dpkg-buildpackage: binary only upload (no source included)
dpkg-source: info: using options from src/debian/source/options: --compression=xz
* SKIP no tests in this package
autopkgtest [14:24:14]: @@@@@@@@@@@@@@@@@@@@ summary
* SKIP no tests in this package

autopkgtest is able to continue with -proposed dpkg.

tags: added: verification-done verification-done-trusty
removed: verification-needed verification-needed-trusty
Witold Szczeponik (wsz) wrote :

I can confirm that the proposed package fixes the problem I reported (with sample ubuntu-minimal_1.412_amd64.deb being from bionic's repositories):

$ dpkg-deb -W ./ubuntu-minimal_1.412_amd64.deb
ubuntu-minimal 1.412

$ lsb_release -rd
Description: Ubuntu 14.04.5 LTS
Release: 14.04

$ apt-cache policy dpkg
dpkg:
  Installed: 1.17.5ubuntu5.8
  Candidate: 1.17.5ubuntu5.8
  Version table:
 *** 1.17.5ubuntu5.8 0
        100 /var/lib/dpkg/status
     1.17.5ubuntu5.7 0
        500 http://archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
     1.17.5ubuntu5.6 0
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
     1.17.5ubuntu5 0
        500 http://archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

Witold Szczeponik (wsz) wrote :

I can also confirm that building a "Bionic" debootstrap environment on "Trusty" works too.

cleary (bernard-gray) wrote :

@sil2100 Tested, confirmed it's fixed this bug for me - thankyou :)

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpkg - 1.17.5ubuntu5.8

---------------
dpkg (1.17.5ubuntu5.8) trusty; urgency=medium

  * Add support for .deb archives with a control member not compressed
    (control.tar) or compressed with xz (control.tar.xz) LP: #1730627.

 -- Adam Conrad <email address hidden> Mon, 04 Dec 2017 12:15:45 -0700

Changed in dpkg (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for dpkg has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Phil Evans (quantumphil) wrote :

I'm seeing this bug on Ubuntu bionic 18.04 with dpkg v.1.19.0.5ubuntu2.

$ lsb_release -rd
Description: Ubuntu 18.04 LTS
Release: 18.04

$ sudo apt-cache policy dpkg
dpkg:
  Installed: 1.19.0.5ubuntu2
  Candidate: 1.19.0.5ubuntu2
  Version table:
 *** 1.19.0.5ubuntu2 500
        500 http://<internal repo pointing to us.rsync.archive.ubuntu.com/ubuntu/>/ubuntu bionic/main amd64 Packages
        100 /var/lib/dpkg/status

$ sudo dpkg -i <file>.deb
dpkg-deb: error: archive '<file>.deb' has premature member 'data.tar.gz' before 'control.tar', giving up
dpkg: error processing archive <file>.deb (--install):
 dpkg-deb --control subprocess returned error exit status 2
Errors were encountered while processing:
 <file>.deb

Dan Streetman (ddstreet) wrote :

> I'm seeing this bug on Ubuntu bionic 18.04 with dpkg v.1.19.0.5ubuntu2.

no, you're not. The message may look similar but it is not the same.

> $ sudo dpkg -i <file>.deb

hm, i don't have the '<file>.deb' anywhere on my system ;-P

Obfuscating what deb you're have a problem with isn't helpful as the problem very well may be with that deb, and makes it impossible for anyone to reproduce your problem.

I'm having the same issue with the package check-mk-agent_1.5.0b9-1_all.deb from Nagios (Check_MK).

# lsb_release -rd
Description: Ubuntu 18.04.1 LTS
Release: 18.04

# apt-cache policy dpkg
dpkg:
  Installed: 1.19.0.5ubuntu2.1
  Candidate: 1.19.0.5ubuntu2.1
  Version table:
 *** 1.19.0.5ubuntu2.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.19.0.5ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages

# dpkg -i check-mk-agent_1.5.0b9-1_all.deb
(Reading database ... 239815 files and directories currently installed.)
Preparing to unpack check-mk-agent_1.5.0b9-1_all.deb ...
Unpacking check-mk-agent (1.5.0b9-1) ...
dpkg-deb: error: archive 'check-mk-agent_1.5.0b9-1_all.deb' contains two control members, giving up
dpkg: error processing archive check-mk-agent_1.5.0b9-1_all.deb (--install):
 dpkg-deb --fsys-tarfile subprocess returned error exit status 2
Reloading xinetd...
Errors were encountered while processing:
 check-mk-agent_1.5.0b9-1_all.deb

Dan Streetman (ddstreet) wrote :

> I'm having the same issue with the package check-mk-agent_1.5.0b9-1_all.deb from
> Nagios (Check_MK).

The problem is with your deb, not dpkg. The check-mk-agent deb from Ubuntu works fine:

$ apt-cache policy check-mk-agent
check-mk-agent:
  Installed: 1.2.8p16-1ubuntu0.1
  Candidate: 1.2.8p16-1ubuntu0.1
  Version table:
 *** 1.2.8p16-1ubuntu0.1 500
        500 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
        100 /var/lib/dpkg/status

$ sudo dpkg -i check-mk-agent_1.2.8p16-1ubuntu0.1_amd64.deb
Selecting previously unselected package check-mk-agent.
(Reading database ... 40378 files and directories currently installed.)
Preparing to unpack check-mk-agent_1.2.8p16-1ubuntu0.1_amd64.deb ...
Unpacking check-mk-agent (1.2.8p16-1ubuntu0.1) ...
Setting up check-mk-agent (1.2.8p16-1ubuntu0.1) ...

This is not the bug you are looking for.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers