docker tarballs contain /dev/null

[Michael Hudson-Doyle]

[impact]

This causes problems for some users as reported in https://github.com/tianon/docker-brew-ubuntu-core/issues/62. We don't want to regress those users.

[test case]
Build a livefs with PROJECT=ubuntu-base SUBPROJECT=minimized. Check that /dev is empty in the resulting tarball.

[regression potential]
The change pretty clearly only affects ubuntu-base:minimized builds, so checking that all of /var/cache/apt, /var/lib/apt/lists and /dev are empty should be sufficient, no other build types should regress.

[Michael Hudson-Doyle (mwhudson)]

Uploads done for all series. Turns out we never backported the change that regressed here to bionic but I uploaded the new version of the fix to bionic UNAPPROVED anyway, it's definitely something we want done there in general.

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 2.581

---------------
livecd-rootfs (2.581) eoan; urgency=medium

  * Remove device nodes later for ubuntu-base:minimized (i.e. docker) builds.
    (LP: #1828118)

 -- Michael Hudson-Doyle <email address hidden> Wed, 08 May 2019 09:59:35 +1200

[Brian Murray (brian-murray)]

Hello Michael, or anyone else affected,

Accepted livecd-rootfs into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.578.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Michael, or anyone else affected,

Accepted livecd-rootfs into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.542.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Michael, or anyone else affected,

Accepted livecd-rootfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.525.23 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Michael Hudson-Doyle (mwhudson)]

In what follows, the ubuntu-${series}-core-cloudimg-amd64-root.tar.gz tarball is the current tarball from https://partner-images.canonical.com/core/ and livecd.ubuntu-base.rootfs-${series}.tar.gz is a test build I made with -proposed enabled (see build pages at https://launchpad.net/~mwhudson/+livefs/ubuntu/disco/test/+build/165334 https://launchpad.net/~mwhudson/+livefs/ubuntu/cosmic/test/+build/165335 https://launchpad.net/~mwhudson/+livefs/ubuntu/bionic/test/+build/165336):

mwhudson@ringil:~/Downloads$ diff -u <(cat ubuntu-bionic-core-cloudimg-amd64-root.tar.gz | tar -tz | sort) <(cat livecd.ubuntu-base.rootfs-bionic.tar.gz | tar -tz | sort)
--- /dev/fd/63 2019-05-10 11:37:29.145681446 +1200
+++ /dev/fd/62 2019-05-10 11:37:29.149681443 +1200
@@ -86,19 +86,6 @@
 bin/znew
 boot/
 dev/
-dev/fd
-dev/full
-dev/null
-dev/ptmx
-dev/pts/
-dev/random
-dev/shm/
-dev/stderr
-dev/stdin
-dev/stdout
-dev/tty
-dev/urandom
-dev/zero
 etc/
 etc/adduser.conf
 etc/alternatives/
@@ -2812,24 +2799,6 @@
 var/lib/apt/
 var/lib/apt/extended_states
 var/lib/apt/lists/
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_InRelease
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_main_binary-amd64_Packages
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_main_i18n_Translation-en
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_restricted_binary-amd64_Packages
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_restricted_i18n_Translation-en
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic-updates_InRelease
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic-updates_main_binary-amd64_Packages
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic-updates_main_i18n_Translation-en
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic-updates_restricted_binary-amd64_Packages
-var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic-updates_restricted_i18n_Translation-en
-var/lib/apt/lists/auxfiles/
-var/lib/apt/lists/lock
-var/lib/apt/lists/partial/
-var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_bionic-security_InRelease
-var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_bionic-security_main_binary-amd64_Packages
-var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_bionic-security_main_i18n_Translation-en
-var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_bionic-security_restricted_binary-amd64_Packages
-var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_bionic-security_restricted_i18n_Translation-en
 var/lib/apt/mirrors/
 var/lib/apt/mirrors/partial/
 var/lib/apt/periodic/
mwhudson@ringil:~/Downloads$ diff -u <(cat ubuntu-cosmic-core-cloudimg-amd64-root.tar.gz | tar -tz | sort) <(cat livecd.ubuntu-base.rootfs-cosmic.tar.gz | tar -tz | sort)
--- /dev/fd/63 2019-05-10 11:37:37.853671457 +1200
+++ /dev/fd/62 2019-05-10 11:37:37.853671457 +1200
@@ -86,7 +86,6 @@
 bin/znew
 boot/
 dev/
-dev/null
 etc/
 etc/adduser.conf
 etc/alternatives/
@@ -2788,9 +2787,6 @@
 var/lib/apt/
 var/lib/apt/extended_states
 var/lib/apt/lists/
-var/lib/apt/lists/auxfiles/
-var/lib/apt/lists/lock
-var/lib/apt/lists/partial/
 var/lib/apt/mirrors/
 var/lib/apt/mirrors/partial/
 var/lib/apt/periodic/
mwhudson@ringil:~/Downloads$ d...

[Julian Andres Klode (juliank)]

I really do not want you to delete the auxfiles, partial directories and the lock file inside /var/lib/apt/lists, they should still be there.

[Julian Andres Klode (juliank)]

Namely partial/ is an _apt owned sandbox arena. I think there is some code to restore a missing one and fixup permissions but that's super ugly.

The correct change to make for cleaning apt would be to run apt-get clean with -o dir::etc::sourcelist=/dev/null, same with sourceparts.

That will take care of cleaning everything up that should be cleaned up and stop messing with internals.

[Michael Hudson-Doyle (mwhudson)]

Well ok but those are currently deleted downstream anyway by
https://github.com/tianon/docker-brew-ubuntu-core/blob/master/update.sh#L91.
Do you want to submit a patch there and then we can patch it here too? In
the mean time I still would like to release this change.

On Fri, 10 May 2019, 19:30 Julian Andres Klode, <email address hidden>
wrote:

> Namely partial/ is an _apt owned sandbox arena. I think there is some
> code to restore a missing one and fixup permissions but that's super
> ugly.
>
> The correct change to make for cleaning apt would be to run apt-get
> clean with -o dir::etc::sourcelist=/dev/null, same with sourceparts.
>
> That will take care of cleaning everything up that should be cleaned up
> and stop messing with internals.
>
> --
> You received this bug notification because you are a member of Canonical
> Cloudware, which is subscribed to livecd-rootfs in Ubuntu.
> https://bugs.launchpad.net/bugs/1828118
>
> Title:
> docker tarballs contain /dev/null
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/1828118/+subscriptions
>

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 2.525.23

---------------
livecd-rootfs (2.525.23) bionic; urgency=medium

  * Backport two minimizations for the docker images: remove apt lists that
    are removed downstream anyway, and remove device nodes from the image.
    (LP: #1828118)

livecd-rootfs (2.525.22) bionic; urgency=medium

  * Subiquity specific changes SRU LP: #1827357
    - subiquity: make subiquity_config.mount optional
    - Make serial-subiquity@ use the same codepath as tty1 subiquity.
    - Fix ubuntu-server-live images to generate initrd with casper UUID.

 -- Michael Hudson-Doyle <email address hidden> Wed, 08 May 2019 10:48:39 +1200

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 2.542.3

---------------
livecd-rootfs (2.542.3) cosmic; urgency=medium

  * Remove device nodes later for ubuntu-base:minimized (i.e. docker) builds.
    (LP: #1828118)

 -- Michael Hudson-Doyle <email address hidden> Wed, 08 May 2019 10:48:39 +1200

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 2.578.2

---------------
livecd-rootfs (2.578.2) disco; urgency=medium

  * Remove device nodes later for ubuntu-base:minimized (i.e. docker) builds.
    (LP: #1828118)

 -- Michael Hudson-Doyle <email address hidden> Wed, 08 May 2019 09:59:35 +1200