Call germinate with -updates

[Jean-Baptiste Lallement]

In auto/config germinate is called with the release pocket only:
 (cd config/germinate-output && germinate --no-rdepends --no-installer \
                -S $SEEDMIRROR -m $MIRROR -d $SUITE -s $SEED \
                ${COMPONENTS:+-c "$COMPONENTS"} -a $ARCH)

In LTSes some packages are in universe in the release pocket and in main in the update pocket.

For example in focal:

$ rmadison -s focal,focal-updates realmd adcli
 realmd | 0.16.3-3 | focal/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 realmd | 0.16.3-3 | focal-updates | source, amd64, arm64, armhf, i386, ppc64el, riscv64, s390x
 adcli | 0.9.0-1 | focal/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 adcli | 0.9.0-1 | focal-updates | i386
 adcli | 0.9.0-1ubuntu0.20.04.1 | focal-updates | source, amd64, arm64, armhf, ppc64el, riscv64, s390x

When these packages are seeded but they have no reverse dependencies, for instances seeded in the live seed and only main and restricted are considered (for Ubuntu) then these packages are ignored (unknown) by germinate and not added to the live session.

Here is an excerpt from the build log of Focal Ubuntu Deskop (https://launchpadlibrarian.net/530408277/buildlog_ubuntu_focal_amd64_ubuntu_BUILDING.txt.gz)

Downloading http://ftpmaster.internal/ubuntu/dists/focal/main/binary-amd64/Packages.xz file ...
Decompressing http://ftpmaster.internal/ubuntu/dists/focal/main/binary-amd64/Packages.xz file ...
Downloading http://ftpmaster.internal/ubuntu/dists/focal/main/source/Sources.xz file ...
Decompressing http://ftpmaster.internal/ubuntu/dists/focal/main/source/Sources.xz file ...
Downloading http://ftpmaster.internal/ubuntu/dists/focal/restricted/binary-amd64/Packages.xz file ...
Decompressing http://ftpmaster.internal/ubuntu/dists/focal/restricted/binary-amd64/Packages.xz file ...
Downloading http://ftpmaster.internal/ubuntu/dists/focal/restricted/source/Sources.xz file ...
Decompressing http://ftpmaster.internal/ubuntu/dists/focal/restricted/source/Sources.xz file ...
* Downloading http://archive-team.internal/seeds/ubuntu.focal/STRUCTURE
[...]
! Duplicated seed: ntfs-3g
? Unknown live package: realmd
? Unknown live package: adcli
! Duplicated seed: dirmngr
[...]

Germinate should be called with "-d ${SUITE},${SUITE}-updates" to consider both pockets.

[Iain Lane (laney)]

Please see https://code.launchpad.net/~laney/ubuntu-archive-publishing/updates/+merge/402698

I believe this is the kind of thing we need to fix this problem properly, generating Task fields in the archive for -updates - currently that does not happen. See add_task() in livecd-rootfs for how they are used. There might be consequences to doing it that I've not thought of, so maybe this will take a while to land in the archive.

Some workarounds / alternatives which could be done instead, to solve the immediate problem

  - Directly install the needed packages in livecd-rootfs (hardcoding)
  - Make ubiquity or something else in the live task depend on them
  - Add a new package ubuntu-live-meta (example name) generated from the live task and install this, make it work with the normal seed ./update script and then SRUs will pick it up. That's a more general variant on the previous item.

[Jean-Baptiste Lallement (jibel)]

Thanks for your advice. As a workaround we'll try to make the packages a dependency of Ubiquity (or ubiquity-gtk since the UI only exist for this UI)

[Jean-Baptiste Lallement (jibel)]

We tried the workaround to add the packages as dependencies of Ubiquity and rebuilding an ISO with the new Ubiquity and the dependencies pulled automatically. Unfortunately, the result is not what is expected.

The packages are installed in the live session and the target file system.
AD is configured by Ubiquity as expected during installation.
The packages are marked to be kept after installation.

But at the end of the installation, the packages are removed then reinstalled by the installer. As a consequence the configuration done during installation is reset.

We tried to modify script/plugininstall.py and scripts/install.py to force keep the packages by they are still considered garbage by apt and removed.

Finally we added a list of packages to keep so they are not considered to "autopurge" by Ubiquity, the packages are kept as expected by not their dependencies even if they are strong one. The resulting installation is completely inconsistent. Using this approach would mean to manually calculate the list of dependencies of the packages to keep and mark them as keep. This is clearly not something maintainable.

[Brian Murray (brian-murray)]

I'm a bit confused by the current state of this. I booted an Ubuntu Live image (date 20210819) and both adcli and realmd were available in the live environment. Additionally, I performed and install and configured active directory (although I don't have an AD environment) and after booting the installed system realmd and sssd-ad were installed. However, adcli was not on the installed system. I don't if that was because it failed to authenticate or because the package is not necessary.

Regardless, is there anything that still needs fixing for the active directory use case for the 20.04.4 point release which is imminent? (I understand there is still a general problem here that needs fixing.) Thanks!

[Steve Langasek (vorlon)]

live-build/auto/config:

  echo "I: Adding dependencies for Active Directory support (Workaround LP: #1921862)"

  apt-get -y update

  apt-get -y install sssd realmd adcli krb5-config

So it's not broken currently, just horrible.

[Brian Murray (brian-murray)]

Given that the immediate problem (missing packages for active directory) is resolved for 20.04. I'm setting this task to Fix Released for Focal and I've added a task for Jammy so that we can try and do the right thing instead of hard coding packages to install.

[Launchpad Janitor (janitor)]

This bug was fixed in the package livecd-rootfs - 24.04.7

---------------
livecd-rootfs (24.04.7) noble; urgency=medium

  [ Dave Jones ]
  * Remove more bits related to pre-installed desktop images (LP: #2038098)

  [ Jean-Baptiste Lallement ]
  * Include -updates when calling germinate. LP: #1921862.

  [ Dan Bungert ]
  * cpc/vagrant: fix possible autopkgtest hang on ssh-keygen

  [ Simon Poirier ]
  * add systemd-resolved back to buildd images. LP: #2007419.

  [ Steve Langasek ]
  * live-build/functions: add additional debugging to mount_image() on
    failure since we again have loop partitions failing to be block devices.

 -- Steve Langasek <email address hidden> Sun, 03 Dec 2023 21:38:27 -0800