buildd images have a very plain sources.list

[Sergio Schvezov]

While testing out the buildd images on https://cloud-images.ubuntu.com/buildd/daily noticed that the sources.list does not contain all the entries one would usually find

$ lxc exec selected-longhorn bash
croot@INVALID:~# cat /etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse
root@INVALID:~# ls /etc/apt/sources.list.d/
root@INVALID:~#

It should ideally contain -updates and -security

I have discussed with wgrant as this image is also used on launchpad and got confirmation that launchpad replaces those contents completely and this change should not affect them. Reconfirm as you must.

[Colin Watson (cjwatson)]

Adding more things to the sources.list that's put into place just before finalising the image indeed wouldn't affect Launchpad. However, we must be careful that the actual set of packages in the image doesn't include post-release pockets, as that would impair our ability to accurately build packages that e.g. are supposed to be built without -updates.

That is, the process needs to involve building chroots without -security, -updates, etc., and if we put -security and -updates in place in the final sources.list then that has to be done at the end. live-build has various facilities for doing this kind of thing but I don't remember the exact details.

[David Krauser (davidkrauser)]

The existing buildd images will continue to include only packages from the release pocket.

The bootable buildd images, however, will be changed to include packages from the updates and security pockets.

[David Krauser (davidkrauser)]

Hey cjwatson,

Digging into this more, while our buildd images are populated only with packages from the release pocket, they include -updates and -security in their `/etc/apt/sources.list`.

Is this expected or desired?

[Colin Watson (cjwatson)]

Launchpad always puts exactly the sources.list it wants to use in place before doing pretty much anything else with the image, so it makes no difference to us.

IIRC we decided that having -updates and -security there was useful for snapcraft.

[David Krauser (davidkrauser)]

Focal images are configured to use -updates and -security; however, bionic images are not.

Bionic images will be updated to match.

[David Krauser (davidkrauser)]

sergiusens and cjwatson,

after more digging, I've found:

- The bootable buildd image work that recently merged into the
  ubuntu/master (focal) branch inadvertently enabled the updates and
  security pockets in /etc/apt/sources.list

  - By switching the image format from 'none' to 'ext4', we now hit this
    block in livecd-rootfs:

    https://git.launchpad.net/livecd-rootfs/tree/live-build/functions?id=8f76e539b1c8f4473dbf4627a0961bdb3732c7c6#n771

- Once the bootable buildd image work is backported to bionic, the
  updates and security pockets will be enabled there, too.

- Even with these changes, the bootable images are built only with
  packages from the release pocket. If we'd like to pull in the latest
  packages from updates or security, we could do something like the
  following attached MP:

  https://code.launchpad.net/~davidkrauser/livecd-rootfs/+git/livecd-rootfs/+ref/buildd-image-perform-upgrade

  - Is this something that we want to do? Some of my thoughts:

    - Snapcraft is the only group consuming the bootable images, and if
      snaps are always built with updates, this will save a lot of time.

    - The original release plan included gating bootable images on
      launchpad giving a +1 on the lxd tarballs. That will be less
      useful if the images have widely different contents.