curtin/maas don't support multiple (derived) archives/repositories with custom keys

[Paolo de Rosa]

[Impact]

 * Curtin doesn't support multiple derived archive/repositories with
   custom keys as typically deployed in an offline Landscape deployment.
   Adding the custom key resulted in an error when processing the
   apt_source configuration as provided in this setup.

   Curtin has been updated to support the updated apt-source model
   implemented in cloud-init as well. Together the existing Landscape
   deployments for offline users can now supply an apt-source config
   that updates curtin to use the specified derived repository with a
   custom key.

[Test Case]

 * Install proposed curtin package and deploy a system behind a
   Landscape Offline configuration with a derived repo.

  PASS: Curtin will successfully accept the derived repo and install the
        system from the specified apt repository.

  FAIL: Curtin will fail to install the OS with an error like:

  W: GPG error: http://100.107.231.166 trusty InRelease:
  The following signatures couldn't be verified because the public key
  is not available: NO_PUBKEY 2C6F2731D2B38BD3
  E: There are problems and -y was used without --force-yes

  Unexpected error while running command.
  Command: ['chroot', '/tmp/tmpcEfTLw/target', 'eatmydata', 'apt-get',
            '--quiet', '--assume-yes',
            '--option=Dpkg::options::=--force-unsafe-io',
            '--option=Dpkg::Options::=--force-confold', 'install',
            'lvm2', 'ifenslave']
  Exit code: 100

[Regression Potential]

 * Other users of previous curtin 'apt_source' configurations may not
   continue to work without re-formatting the apt_source configuration.

[Original Description]

In a customer environment I have to deploy using offline resources (no internet connection at all), so I created apt mirror and MAAS images mirror. I configured MAAS to use the local mirrors and I'm able to commission the nodes but I'm not able to deploy the nodes because there is no way to add gpg key of the local repo in target before the 'late' stage'.

Using curtin I'm able to add the key but too late, in fact according with http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/curtin/commands/install.py#L52 "late" stage is executed after "curthooks" this prevent to add the key.

I checked also apt_config function in curthooks.py I did't see code that add the key for each mirror.

It should be possible to add gpg public of the repository in maas.

----------------------------------
configs/config-000.cfg
----------------------------------

#cloud-config
debconf_selections:
 maas: |
  cloud-init cloud-init/datasources multiselect MAAS
  cloud-init cloud-init/maas-metadata-url string http://100.107.231.164/MAAS/metadata/
  cloud-init cloud-init/maas-metadata-credentials string oauth_token_key=8eZmzQWSSQzsUkaLnE&oauth_token_secret=LKmn8sHgzEXfvzSZePAa9jUXvTMRrFNP&oauth_consumer_key=htwDZJFtmv2YvQXhUW
  cloud-init cloud-init/local-cloud-config string apt_preserve_sources_list: true\nmanage_etc_hosts: false\nmanual_cache_clean: true\nreporting:\n maas: {consumer_key: htwDZJFtmv2YvQXhUW, endpoint: 'http://100.107.231.164/MAAS/metadata/status/node-61b6987c-07a7-11e6-9d23-5254003d2515',\n token_key: 8eZmzQWSSQzsUkaLnE, token_secret: LKmn8sHgzEXfvzSZePAa9jUXvTMRrFNP,\n type: webhook}\nsystem_info:\n package_mirrors:\n - arches: [i386, amd64]\n failsafe: {primary: 'http://archive.ubuntu.com/ubuntu', security: 'http://security.ubuntu.com/ubuntu'}\n search:\n primary: ['http://100.107.231.166/']\n security: ['http://100.107.231.166/']\n - arches: [default]\n failsafe: {primary: 'http://ports.ubuntu.com/ubuntu-ports', security: 'http://ports.ubuntu.com/ubuntu-ports'}\n search:\n primary: ['http://ports.ubuntu.com/ubuntu-ports']\n security: ['http://ports.ubuntu.com/ubuntu-ports']\n
late_commands:
  maas: [wget, '--no-proxy', 'http://100.107.231.164/MAAS/metadata/latest/by-id/node-61b6987c-07a7-11e6-9d23-5254003d2515/', '--post-data', 'op=netboot_off', '-O', '/dev/null']
  apt_key: ["curtin", "in-target", "--", "sh", "-c", "/usr/bin/wget --no-proxy -qO - http://100.107.231.166/magellan.key | apt-key add -"]
power_state:
  mode: reboot
apt_mirrors:
  ubuntu_archive: http://100.107.231.166//
  ubuntu_security: http://100.107.231.166//

----- curtin end of log ------
Leaving 'diversion of /etc/init/ureadahead.conf to /etc/init/ureadahead.conf.disabled by cloud-init'
Setting up swapspace version 1, size = 8388604 KiB
no label, UUID=e2fe91bc-91e9-4e43-b50f-209dfcf04089
Get:1 http://100.107.231.166 trusty InRelease [17.7 kB]
Get:2 http://100.107.231.166 trusty-updates InRelease [17.7 kB]
Get:3 http://100.107.231.166 trusty-security InRelease [17.7 kB]
Ign http://100.107.231.166 trusty InRelease
Get:4 http://100.107.231.166 trusty/main amd64 Packages [412 kB]
Ign http://100.107.231.166 trusty-updates InRelease
Ign http://100.107.231.166 trusty-security InRelease
Get:5 http://100.107.231.166 trusty/restricted amd64 Packages [20 B]
Get:6 http://100.107.231.166 trusty/universe amd64 Packages [20 B]
Get:7 http://100.107.231.166 trusty/multiverse amd64 Packages [20 B]
Get:8 http://100.107.231.166 trusty-updates/main amd64 Packages [33.0 kB]
Get:9 http://100.107.231.166 trusty-updates/restricted amd64 Packages [20 B]
Get:10 http://100.107.231.166 trusty-updates/universe amd64 Packages [20 B]
Get:11 http://100.107.231.166 trusty-updates/multiverse amd64 Packages [20 B]
Get:12 http://100.107.231.166 trusty-security/main amd64 Packages [6,578 B]
Get:13 http://100.107.231.166 trusty-security/restricted amd64 Packages [20 B]
Get:14 http://100.107.231.166 trusty-security/universe amd64 Packages [20 B]
Get:15 http://100.107.231.166 trusty-security/multiverse amd64 Packages [20 B]
Fetched 505 kB in 0s (3,772 kB/s)
Reading package lists...
W: GPG error: http://100.107.231.166 trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3
W: GPG error: http://100.107.231.166 trusty-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3
W: GPG error: http://100.107.231.166 trusty-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  libdevmapper-event1.02.1 libreadline5 watershed
Suggested packages:
  thin-provisioning-tools
The following NEW packages will be installed:
  ifenslave libdevmapper-event1.02.1 libreadline5 lvm2 watershed
0 upgraded, 5 newly installed, 0 to remove and 10 not upgraded.
Need to get 635 kB of archives.
After this operation, 1,885 kB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  libdevmapper-event1.02.1 libreadline5 ifenslave watershed lvm2
E: There are problems and -y was used without --force-yes
Unexpected error while running command.
Command: ['chroot', '/tmp/tmpcEfTLw/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'lvm2', 'ifenslave']
Exit code: 100
Reason: -
Stdout: ''
Stderr: ''
builtin command failed
Installation failed with exception: Unexpected error while running command.
Command: ['curtin', 'curthooks']
Exit code: 3
Reason: -
Stdout: "Leaving 'diversion of /etc/init/ureadahead.conf to /etc/init/ureadahead.conf.disabled by cloud-init'\nSetting up swapspace version 1, size = 8388604 KiB\nno label, UUID=e2fe91bc-91e9-4e43-b50f-209dfcf04089\nGet:1 http://100.107.231.166 trusty InRelease [17.7 kB]\nGet:2 http://100.107.231.166 trusty-updates InRelease [17.7 kB]\nGet:3 http://100.107.231.166 trusty-security InRelease [17.7 kB]\nIgn http://100.107.231.166 trusty InRelease\nGet:4 http://100.107.231.166 trusty/main amd64 Packages [412 kB]\nIgn http://100.107.231.166 trusty-updates InRelease\nIgn http://100.107.231.166 trusty-security InRelease\nGet:5 http://100.107.231.166 trusty/restricted amd64 Packages [20 B]\nGet:6 http://100.107.231.166 trusty/universe amd64 Packages [20 B]\nGet:7 http://100.107.231.166 trusty/multiverse amd64 Packages [20 B]\nGet:8 http://100.107.231.166 trusty-updates/main amd64 Packages [33.0 kB]\nGet:9 http://100.107.231.166 trusty-updates/restricted amd64 Packages [20 B]\nGet:10 http://100.107.231.166 trusty-updates/universe amd64 Packages [20 B]\nGet:11 http://100.107.231.166 trusty-updates/multiverse amd64 Packages [20 B]\nGet:12 http://100.107.231.166 trusty-security/main amd64 Packages [6,578 B]\nGet:13 http://100.107.231.166 trusty-security/restricted amd64 Packages [20 B]\nGet:14 http://100.107.231.166 trusty-security/universe amd64 Packages [20 B]\nGet:15 http://100.107.231.166 trusty-security/multiverse amd64 Packages [20 B]\nFetched 505 kB in 0s (3,772 kB/s)\nReading package lists...\nW: GPG error: http://100.107.231.166 trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3\nW: GPG error: http://100.107.231.166 trusty-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3\nW: GPG error: http://100.107.231.166 trusty-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3\nReading package lists...\nBuilding dependency tree...\nReading state information...\nThe following extra packages will be installed:\n libdevmapper-event1.02.1 libreadline5 watershed\nSuggested packages:\n thin-provisioning-tools\nThe following NEW packages will be installed:\n ifenslave libdevmapper-event1.02.1 libreadline5 lvm2 watershed\n0 upgraded, 5 newly installed, 0 to remove and 10 not upgraded.\nNeed to get 635 kB of archives.\nAfter this operation, 1,885 kB of additional disk space will be used.\nWARNING: The following packages cannot be authenticated!\n libdevmapper-event1.02.1 libreadline5 ifenslave watershed lvm2\nE: There are problems and -y was used without --force-yes\nUnexpected error while running command.\nCommand: ['chroot', '/tmp/tmpcEfTLw/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'lvm2', 'ifenslave']\nExit code: 100\nReason: -\nStdout: ''\nStderr: ''\n"
Stderr: ''
failed posting event: finish: cmd-install: FAIL: curtin command install [[http://100.107.231.164/MAAS/metadata/status/node-61b6987c-07a7-11e6-9d23-5254003d2515] http error: 400]
Unexpected error while running command.
Command: ['curtin', 'curthooks']
Exit code: 3
Reason: -
Stdout: "Leaving 'diversion of /etc/init/ureadahead.conf to /etc/init/ureadahead.conf.disabled by cloud-init'\nSetting up swapspace version 1, size = 8388604 KiB\nno label, UUID=e2fe91bc-91e9-4e43-b50f-209dfcf04089\nGet:1 http://100.107.231.166 trusty InRelease [17.7 kB]\nGet:2 http://100.107.231.166 trusty-updates InRelease [17.7 kB]\nGet:3 http://100.107.231.166 trusty-security InRelease [17.7 kB]\nIgn http://100.107.231.166 trusty InRelease\nGet:4 http://100.107.231.166 trusty/main amd64 Packages [412 kB]\nIgn http://100.107.231.166 trusty-updates InRelease\nIgn http://100.107.231.166 trusty-security InRelease\nGet:5 http://100.107.231.166 trusty/restricted amd64 Packages [20 B]\nGet:6 http://100.107.231.166 trusty/universe amd64 Packages [20 B]\nGet:7 http://100.107.231.166 trusty/multiverse amd64 Packages [20 B]\nGet:8 http://100.107.231.166 trusty-updates/main amd64 Packages [33.0 kB]\nGet:9 http://100.107.231.166 trusty-updates/restricted amd64 Packages [20 B]\nGet:10 http://100.107.231.166 trusty-updates/universe amd64 Packages [20 B]\nGet:11 http://100.107.231.166 trusty-updates/multiverse amd64 Packages [20 B]\nGet:12 http://100.107.231.166 trusty-security/main amd64 Packages [6,578 B]\nGet:13 http://100.107.231.166 trusty-security/restricted amd64 Packages [20 B]\nGet:14 http://100.107.231.166 trusty-security/universe amd64 Packages [20 B]\nGet:15 http://100.107.231.166 trusty-security/multiverse amd64 Packages [20 B]\nFetched 505 kB in 0s (3,772 kB/s)\nReading package lists...\nW: GPG error: http://100.107.231.166 trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3\nW: GPG error: http://100.107.231.166 trusty-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3\nW: GPG error: http://100.107.231.166 trusty-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2C6F2731D2B38BD3\nReading package lists...\nBuilding dependency tree...\nReading state information...\nThe following extra packages will be installed:\n libdevmapper-event1.02.1 libreadline5 watershed\nSuggested packages:\n thin-provisioning-tools\nThe following NEW packages will be installed:\n ifenslave libdevmapper-event1.02.1 libreadline5 lvm2 watershed\n0 upgraded, 5 newly installed, 0 to remove and 10 not upgraded.\nNeed to get 635 kB of archives.\nAfter this operation, 1,885 kB of additional disk space will be used.\nWARNING: The following packages cannot be authenticated!\n libdevmapper-event1.02.1 libreadline5 ifenslave watershed lvm2\nE: There are problems and -y was used without --force-yes\nUnexpected error while running command.\nCommand: ['chroot', '/tmp/tmpcEfTLw/target', 'eatmydata', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', 'lvm2', 'ifenslave']\nExit code: 100\nReason: -\nStdout: ''\nStderr: ''\n"
Stderr: ''

[Andres Rodriguez (andreserl)]

HI Paolo,

Why don't you use a 'early_command' instead of 'late_command'?

That being said, even if the key is not available I'd have thought that you would have been able to continue to install, but the two/packages were unauthenticated.

[Andres Rodriguez (andreserl)]

Also, how did you create your mirror? IIRC, If you create a Ubuntu mirror you should have the keys you need to voerify on the system itself. I'm thinking that the bug here is that your mirror was not set up properly, hence you experience the key issues.

[Mike Pontillo (mpontillo)]

Why is a separate signing key in use? When I create my mirror, the GPG signatures are inclided as well, and so it verifies properly out-of-the-box. Does the mirror need another signing key because it modifies some official Ubuntu packages? Or can this approach be used?

[Paolo de Rosa (paolo-de-rosa)]

Hi Andres and Mike,

using early_command it's not an option because in that stage the target (the chrooted image ) is not ready, so using early stage I'm able to add the key in the installing system but not in the chroot system (the real one).

To create the mirror I'm using reprepro, the mirror is well configured but I'm not sure that I can avoid the the signature of Release file.

We are talking about the metadata of the mirror not the packages , I mean the metadata of this mirror (Release InRelease etc) are generated and signed by reprepro, I don't need to sign the packages, there are no changes in the packages.
If there is a way to avoid the signature please let me know, I'm ok with that solution for now.

 In later stages we will use landscape to manage the life cycle of the packages and the signature of the packages will be mandatory.

I'm using reprpro because the mirror should be as small as possible in this initial phase, using reprepro I can filter easily the contents of the mirror, it's quite simple to use and stable, but if you have suggestion in order to workaround this problem I'm happy to try valid alternatives.

[Andres Rodriguez (andreserl)]

Can you please share how was your repo configured/created?

[Paolo de Rosa (paolo-de-rosa)]

As temporary solution it could be possible to add something like the snippet in the link above ?

http://bazaar.launchpad.net/~paolo-de-rosa/curtin/curtin/revision/384

thanks
p.

[Andres Rodriguez (andreserl)]

Hi Paolo,

I will raise this with the curtin team on Monday, but to me it seems like a
mis configuration of the mirror. Again, can you please share how you
created and configured the mirror?

On Sunday, April 24, 2016, Paolo de Rosa <email address hidden>
wrote:

> As temporary solution it could be possible to add something like the
> snippet in the link above ?
>
> http://bazaar.launchpad.net/~paolo-de-rosa/curtin/curtin/revision/384
>
> thanks
> p.
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1574113
>
> Title:
> Deploy fails in an offline environment
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/curtin/+bug/1574113/+subscriptions
>

--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

[Mike Pontillo (mpontillo)]

Yes, I think it would be difficult to support reporepo out-of-the-box unless we can supplement the trusted key in the image.

I have had better luck with full rsync mirrors. Also, apt-mirror or debmirror could be used to create smaller mirrors without changing the signing key, though I don't know how powerful the filtering is if you want to exclude specific debs.

Alternatively you could replace the signed Release and InRelease files in reporepo with the "officially signed" files. They should specify the same packages, so it should work fine as long as the files specified are the same.

[Paolo de Rosa (paolo-de-rosa)]

Hi Andres,

what do you want to know exactly ?
I'm using reprepro (below the config files) with a gpg key generated ad-hoc and nginx.

What do you exactly expect to be wrong ?
'Release' file is generated by reprepro and it needs to be signed [1], I generated a key that it's not obviously present in the distribution, it has to be imported, so how the key should be imported by apt system ?

Sorry but from 2006 I followed the rules exposed in [1]. I don't know if there is something newer or a different approach in ubuntu. Could you please point me to some docs/specs where I can study how to build a proper apt repository for MAAS ?

[1] https://wiki.debian.org/SecureApt

=====
this is the directory layout exposed trough nginx:

root@apt-mirror:/srv/repositories# ls
conf db dists lists logs pool pubkeys

pubkyes -> the pub gpg key
conf -> config files for reprepro

the private gpg key has been generated and it's available under root user in the keyring for gpgagent.
Some packages have been imported individually not trough 'reprepro update" process, because they are not included by the filter (priority (==required) | priority (==important)).

I also copied these files, how suggested by Mike:
rsync --recursive --times --links --hard-links --delete --delete-after --verbose \
    rsync://archive.ubuntu.com/ubuntu/dists/trusty/main/uefi/ \
    /srv/repositories/dists/trusty/main/uefi

===== conf/distributions =====
Origin: Ubuntu
Codename: trusty
Description: Ubuntu trusty mirror
Architectures: i386 amd64
Components: main multiverse restricted universe
UDebComponents: main restricted universe multiverse
Contents: .gz
UDebIndices: Packages Release . .gz
Update: - ubuntu-trusty
Log: /srv/repositories/logs/mirror.log
SignWith: D2B38BD3

Origin: Ubuntu
Codename: trusty-updates
Description: Ubuntu trusty updates
Architectures: i386 amd64
Components: main multiverse restricted universe
UDebComponents: main restricted universe multiverse
Contents: .gz
UDebIndices: Packages Release . .gz
Update: - ubuntu-trusty-updates
Log: /srv/repositories/logs/mirror.log
SignWith: D2B38BD3

Origin: Ubuntu
Codename: trusty-security
Description: Ubuntu trusty security
Architectures: i386 amd64
Components: main multiverse restricted universe
UDebComponents: main restricted universe multiverse
Contents: .gz
UDebIndices: Packages Release . .gz
Update: - ubuntu-trusty-security
Log: /srv/repositories/logs/mirror.log
SignWith: D2B38BD3

===== conf/updates =====

Name: ubuntu-trusty
Method: http://archive.ubuntu.com/ubuntu
Components: main multiverse restricted universe
Suite: trusty
UDebComponents: main restricted universe multiverse
Architectures: i386 amd64
FilterFormula: priority (==required) | priority (==important)
VerifyRelease: blindtrust
GetInRelease: no

Name: ubuntu-trusty-security
Method: http://archive.ubuntu.com/ubuntu
Components: main multiverse restricted universe
Suite: trusty-security
UDebComponents: main restricted universe multiverse
Architectures: i386 amd64
FilterFormula: priority (==required) | priority (==important)
VerifyRelease: blindtrust

Name: ubuntu-trusty-updates
Method: http://archive.ubuntu.com/ubuntu
Components: main multivers...

[Mike Pontillo (mpontillo)]

I haven't used reporepo before, but I wonder: if you remove the SignWith line from the configuration, does it retain the existing signatures?

[Paolo de Rosa (paolo-de-rosa)]

If you remove the SignWith option no signature will be done and we have the same problem. Something like [1] 6 lines of code in curtin should solve the problem for the moment, in this way I can add the gpg key as string in curtin_userdata.

[1] http://bazaar.launchpad.net/~paolo-de-rosa/curtin/curtin/revision/384

[LaMont Jones (lamont)]

The only reason to use reprepro would be if you are adding your own packages to the repo. If that is the case, then simply include your key in the appropriate keyring package (ubuntu-cloudimage-keyring or ubuntu-keyring, as appropriate), and you'll be fine. If you aren't modifying any packages in the archive as part of your mirroring, then you should not be re-signing the packages.

[Andres Rodriguez (andreserl)]

Hi Paolo,

Based on LaMont's feedback above, the problem you are having is because of the mirror. You've created a new mirror that you signed yourself with a different key, and in reality, it is not an exact copy of the Ubuntu mirror. As such, curtin will fail to verify packages and fail to install.

The options we now have:

1. Create an actual mirror of the archive, to not experience the issues.
2. Find another way how to inject the key via curtin.

I'll mark the MAAS bug as invalid. And re-name this to see if curtin can add or provide some feedback on the second. However, the recommendation is to create a proper mirror of the archive.

[Andres Rodriguez (andreserl)]

Paolo,

Again, we highly recommend you create a mirror properly to not have this issue, however, Scott Moser suggested this may work if you wanted to inject the key:

# This is an example of a yaml 'anchor' that allows you to reference
# a string which makes typing things more easily read and reduces
# need for escaping. 'myscript' is defined as an anchor here in the
# arbitrarily named 'bucket' and then referenced as the argument to
# sh below via '*myscript'.
bucket:
 - &myscript |
   echo "hi mom. i'm running in your target"

# curtin's 'curthooks' stage installs packages into the target
# the default entry in this dictionary is:
# builtin: [curtin, curthooks]
#
# entries added to the curthooks_commands dictionary are executed
# in C locale sorted order. So to run before 'builtin', you
# need to start with something before that.
curthooks_commands:
  aa_pre_curthooks01: [curtin, in-target, "--", sh, -c, *myscript]

[LaMont Jones (lamont)]

The methods for creating a mirror are well described at https://wiki.ubuntu.com/Mirrors -- If you want to just have the entire mirror, follow the link under https://wiki.ubuntu.com/Mirrors#Country_mirror_requirements to https://wiki.ubuntu.com/Mirrors/Scripts to find the recommended method for a complete mirror.

[Paolo de Rosa (paolo-de-rosa)]

Hi Andres and LaMont,
I think that there is a communication problem , I'm really sorry for my bad English so I'll try to explain again the problem.

As I said before in the posts, I'm not doing a FULL mirror of ubuntu packages. I need to create a PARTIAL mirror of ubuntu packages with only a small subset of packages, as LaMont suggested me I need to install from a DERIVED REPO

Only the minimum packages necessary to install MAAS and the base system, should be present in this new source.

The methods you suggested describe how to create a FULL mirror 926Gb of archive, not what I need.

Thanks a lot for the information and I'll try the solution proposed by Scott Moser

thanks
p.

[Mike Pontillo (mpontillo)]

Paolo, you can safely *remove* packages from the archive without changing the signature of the release files. You must only re-sign the release files if you are *adding* packages. (the client will get an error if it believes the missing packages exist and tries to download them, but that's a separate problem.)

[Ante Karamatić (ivoks)]

Andreas, Mike

I strongly disagree about the state of this bug. If MAAS cannot handle custom keys, then we can't use Landscape as an archive backend. https://landscape.canonical.com/static/doc/user-guide/ch09.html

[Andres Rodriguez (andreserl)]

Curtin would need ot support something like:

https://pastebin.canonical.com/155231/

And then MAAS add the counter part.

[Andres Rodriguez (andreserl)]

The patch for this:

http://bazaar.launchpad.net/~paolo-de-rosa/curtin/curtin/revision/384

[Scott Moser (smoser)]

What i'd suggest to do in curtin is
a.) add support for the 'apt_sources' syntax in cloud-init [1], and change that so that it supports keys without a sources (currently the implementation doesn't do that)
b.) allow some config option to provide a template for /etc/apt/sources.list rather than relying on the one that is builtin.

'a' allows adding keys (even to be used by the default repos) and repos.
'b' would allow maas to provide whatever /etc/apt/sources.list they wanted (even a blank one and rely only on apt_sources definitions).

i may want to improve the syntax for cloud-init and also improve it there to support a dictionary rather than a list. Dictionaries are easier to "merge", and curtin's config merger has good support for that.
So it might look something like:
apt_sources:
  sources:
    smppa: source: ppa:smoser
    localkey0: key: |
      your key here
  sources_list_template:
     some template for sources.list here.

the key thing is that we want to have cloud-init and curtin with the same function.

--
[1] http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L79

[Christian Ehrhardt  (paelzer)]

Hi,
the cloud init portion that is required for this functionality is ready in a MP at https://code.launchpad.net/~paelzer/cloud-init/test-apt-source/+merge/294521 .
I think this should get in soon depending on smoser's next review.

IF/WHAT change we need in curtin and/or maas to exploit these has to be discussed - I couldn't reach smoser last Friday - but in the meantime I think that is something to be discussed with the MAAS team ahead of time anyway.

IMHO - given the cloud-init changes, the MAAS/cloud-init preseeds infrastructure could be already enough for any site that wants to use this feature to set it up with keys and custom source.list templates as they need to allow a derived repositories.

@Andres: If you could set up a call with smoser and me that works for you - that would be great.
IIRC you are currently sprinting and I know combining your current TZ and mine in Germany won't leave a lot of options, but I think you have the busiest schedule atm - so you should be scheduling it.

[Scott Moser (smoser)]

fix-committed in revno 1224.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.7~bzr1227-0ubuntu1

---------------
cloud-init (0.7.7~bzr1227-0ubuntu1) yakkety; urgency=medium

  * New upstream snapshot.
    - fix one more unit test to run inside buildd.

 -- Scott Moser <email address hidden> Sat, 04 Jun 2016 20:55:07 -0400

[Christian Ehrhardt  (paelzer)]

Please be aware of related bug 1589174 that kind of is a post cleanup for the cloud-init portion.

[Scott Moser (smoser)]

Hello,
An SRU upload of cloud-init for 16.04 that contains a fix for this bug has been made under bug 1595302. Please track that bug if you are interested.

[Scott Moser (smoser)]

fix is now released to xenial under bug 1595302. daily cloud-images with this newer version of cloud-init should appear in the next few days. Any image with a serial number *newer* than 20160707 should have cloud-init at 0.7.7~bzr1246-0ubuntu1~16.04.1 .

[Scott Moser (smoser)]

Please strike that last comment.
this is not appropriately fixed in cloud-init in xenial although a related change did go in.

[Scott Moser (smoser)]

This is fixed in cloud-init 0.7.7

[Scott Moser (smoser)]

This is in xenial SRU at the moment.
see package version 0.7.7-31-g65ace7b-0ubuntu1~16.04.1 .

[Andy Whitcroft (apw)]

Hello Paolo, or anyone else affected,

Accepted curtin into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/curtin/0.1.0~bzr425-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Andres Rodriguez (andreserl)]

Verified, marking as verification-done

[Launchpad Janitor (janitor)]

This bug was fixed in the package curtin - 0.1.0~bzr425-0ubuntu1~16.04.1

---------------
curtin (0.1.0~bzr425-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  [ Scott Moser ]
  * debian/new-upstream-snapshot: add writing of debian changelog entries.

  [ Ryan Harper ]
  * New upstream snapshot.
    - unittest,tox.ini: catch and fix issue with trusty-level mock of open
    - block/mdadm: add option to ignore mdadm_assemble errors (LP: #1618429)
    - curtin/doc: overhaul curtin documentation for readthedocs.org
      (LP: #1351085)
    - curtin.util: re-add support for RunInChroot (LP: #1617375)
    - curtin/net: overhaul of eni rendering to handle mixed ipv4/ipv6 configs
    - curtin.block: refactor clear_holders logic into block.clear_holders and
      cli cmd
    - curtin.apply_net should exit non-zero upon exception. (LP: #1615780)
    - apt: fix bug in disable_suites if sources.list line is blank.
    - vmtests: disable Wily in vmtests
    - Fix the unittests for test_apt_source.
    - get CURTIN_VMTEST_PARALLEL shown correctly in jenkins-runner output
    - fix vmtest check_file_strippedline to strip lines before comparing
    - fix whitespace damage in tests/vmtests/__init__.py
    - fix dpkg-reconfigure when debconf_selections was provided.
      (LP: #1609614)
    - fix apt tests on non-intel arch
    - Add apt features to curtin. (LP: #1574113)
    - vmtest: easier use of parallel and controlling timeouts
    - mkfs.vfat: add force flag for formating whole disks (LP: #1597923)
    - block.mkfs: fix sectorsize flag (LP: #1597522)
    - block_meta: cleanup use of sys_block_path and handle cciss knames
      (LP: #1562249)
    - block.get_blockdev_sector_size: handle _lsblock multi result return
      (LP: #1598310)
    - util: add target (chroot) support to subp, add target_path helper.
    - block_meta: fallback to parted if blkid does not produce output
      (LP: #1524031)
    - commands.block_wipe: correct default wipe mode to 'superblock'
    - tox.ini: run coverage normally rather than separately
    - move uefi boot knowledge from launch and vmtest to xkvm

 -- Ryan Harper <email address hidden> Mon, 03 Oct 2016 13:43:54 -0500

[Martin Pitt (pitti)]

The verification of the Stable Release Update for curtin has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Ante Karamatić (ivoks)]

Is there any way to use this functionality in 14.04?

[Christian Ehrhardt  (paelzer)]

On Mon, Nov 7, 2016 at 1:08 PM, Ante Karamatić <<email address hidden>
> wrote:

> Is there any way to use this functionality in 14.04?

Hi Ante,
Since it was not targeted, developed nor tested at 14.04 it can not be used
as-is.
But OTOH the code is rather release agnostic, so 14.04 it might still just
work once you have the right curtin/cloud-init bits available.

But to test that is a bit harder than the usual "add a ppa" action.
You'd need to have 14.04 images that MAAS deploys to have the new code as
well as get MAAS to exploit that.

So for the moment, no - there is no way (yet?) to use this in 14.04

[Scott Moser (smoser)]

This bug is believed to be fixed in curtin in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2658