apt-mirror does not override security.ubuntu.com for containers on trusty

[Nobuto Murata]

With "apt-mirror"[1] config, juju 1.25 can override archive.ubuntu.com with a local mirror. However security.ubuntu.com in sources.list is untouched. To make charm execution successful without Internet connectivity, replacing security.ubuntu.com with local mirror is essential.

As of 1.25.5, juju generates user-data with apt-mirror as:
====
#cloud-config
apt_mirror: http://localmirror.example.com/ubuntu
====

To override security.ubuntu.com juju needs to generate user-data like:
====
#cloud-config
system_info:
  package_mirrors:
    - arches: [i386, amd64]
      failsafe:
        primary: http://localmirror.example.com/ubuntu
        security: http://localmirror.example.com/ubuntu
      search:
        primary: []
        security: []
====

For more information regarding the override, please refer to [2].

[1] https://jujucharms.com/docs/stable/models-config#apt-mirror
[2] https://bugs.launchpad.net/cloud-init/+bug/1006963

[James Tunnicliffe (dooferlad)]

Thanks for the bug report. The apt-mirror setting seems to copy what setting your desktop's apt mirror does in software sources, which isn't useful in your case.

I think the current supported way to isolate a Juju environment from the internet, but still be able to install packages, is to set up a proxy. This is what I have used in the past (but haven't tried recently):

In clouds.yaml for Juju 2.0. Similar options are available for 1.25.x
clouds:
  maas-proxied:
    type: maas
    auth-types: [oauth1]
    endpoint: http://192.168.1.2/MAAS/
    apt-http-proxy: 192.168.0.2:8000
    apt-https-proxy: 192.168.0.2:8000
    http-proxy: 192.168.0.2:8000
    https-proxy: 192.168.0.2:8000

[Nobuto Murata (nobuto)]

Sorry if it was not clear about the intention of this bug report. This is a follow-up of https://bugs.launchpad.net/juju-core/+bug/1560391.

MAAS can already override archive.ubuntu.com and security.ubuntu.com by passing a config like below to curtin:
...
apt_mirrors:
  ubuntu_archive: http://localmirror.example.com/ubuntu
  ubuntu_security: http://localmirror.example.com/ubuntu
...

Juju can override archive.ubuntu.com for containers by apt-mirror in juju config, but cannot override security.ubuntu.com at this moment. `apt-get update` inside container will fail if it does not have connectivity to security.ubuntu.com.

[Seyeong Kim (seyeongkim)]

I tested juju 2.0.2 and 2.1 beta

but it is set -security correctly.

do i need to check other than juju version?

[Nobuto Murata (nobuto)]

Looks like the behavior of cloud-init has been changed between trusty and xenial. So it's no longer critical to xenial deployment, only trusty deployment is affected.

[Seyeong Kim (seyeongkim)]

yeah, I confirmed that trusty is affaced not xenial.

Thanks

[Anastasia (anastasia-macmood)]

Thank you for update \o/

I am removing 2.1 milestone as we will not be addressing this issue in 2.1.

[Seyeong Kim (seyeongkim)]

cloud-init on trusty has no functionality for security repository. (0.7.5-0ubuntu1.21)

below commit on upstream is related but there are a lot of dependencies. so i'm not sure it's fixable.

commit d861415ff9ab816b1183b8c58ec35348be4fd458
Author: Christian Ehrhardt <email address hidden>
Date: Wed Aug 10 16:43:14 2016 +0200

    Apt: add new apt configuration format

    This adds an improved apt configuration format that is fully backwards
    compatible with previous behavior. This is mostly copied from curtin's
    implementation.

    It does:
     * clean up and centralizes many of the top level 'apt_*' values that
       previously existed into a single top level 'apt'key.
     * support a 'source' in apt/sources/entry that has only a key
     * documents new features and adds tests.

    See the added doc/examples/cloud-config-apt.txt for more information.

[Anastasia (anastasia-macmood)]

@Seyeong Kim (xtrusia),

You have previously tested against 2.0.2 and 2.1 beta (I am guessing both on xenial).

Have you had a chance to test your fix with Juju on trusty? Latest equivalent versions are 2.0.3 and 2.1-rc1.

[Nobuto Murata (nobuto)]

As I wrote in the bug description, security.ubuntu.com can be overwritten on trusty by:
====
#cloud-config
system_info:
  package_mirrors:
    - arches: [i386, amd64]
      failsafe:
        primary: http://localmirror.example.com/ubuntu
        security: http://localmirror.example.com/ubuntu
      search:
        primary: []
        security: []
====
instead of apt_* variables.

I'm not sure it's appropriate to treat it as a cloud-init bug.

[Seyeong Kim (seyeongkim)]

tested 2.0.3 but have same issue

and FYI

I talked smoser who maintains cloud-init.

he also think this is not cloud-init bug.

[Seyeong Kim (seyeongkim)]

@Anastasia (anastasia-macmood)

tested 2.0.3, 2.1 and 2.2-alpha1.1 but have same issue.

[John A Meinel (jameinel)]

It feels like overriding security.ubuntu.com should be a separate setting,
since it is security sensitive, vs having the normal 'apt_mirror' override
both settings always. Otherwise why would cloud-init itself have 2
different settings.

maybe 'apt_security_mirror' ? I'm not sure on the specific syntax there.

On Wed, Mar 1, 2017 at 7:57 PM, Seyeong Kim <email address hidden>
wrote:

> @Anastasia (anastasia-macmood)
>
> tested 2.0.3, 2.1 and 2.2-alpha1.1 but have same issue.
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> https://bugs.launchpad.net/bugs/1599886
>
> Title:
> apt-mirror does not override security.ubuntu.com for containers on
> trusty
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1599886/+subscriptions
>

[Seyeong Kim (seyeongkim)]

I agree with jameinel,

supporting kind of apt_security_mirror is better rather than failsafe.

but still there is one issue if juju can support sort of 'apt_security_mirror' from now.

because of differences between cloud-init versions on trusty and xenial.

I'm not sure it's possible to make cloud-init on trusty support apt_seurity_mirror via SRU process.

[John A Meinel (jameinel)]

So Juju itself could accept apt_security_mirror as the configuration key
and map that to whatever is necessary for the series we are deploying.

John
=:->

On Mar 8, 2017 20:19, "Seyeong Kim" <email address hidden> wrote:

> I agree with jameinel,
>
> supporting kind of apt_security_mirror is better rather than failsafe.
>
> but still there is one issue if juju can support sort of
> 'apt_security_mirror' from now.
>
> because of differences between cloud-init versions on trusty and xenial.
>
> I'm not sure it's possible to make cloud-init on trusty support
> apt_seurity_mirror via SRU process.
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> https://bugs.launchpad.net/bugs/1599886
>
> Title:
> apt-mirror does not override security.ubuntu.com for containers on
> trusty
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1599886/+subscriptions
>

[Antti Rahikainen (anrah)]

Any milestone for this?

This is quite crucial when deploying on private clouds and limiting outgoing traffic on instances to only trusted or validated sources. And we don't want to use security.ubuntu.com as repository but our own private repositories inside the OpenStack

[Dmitrii Shcherbakov (dmitriis)]

cloud-init user-data customization seems to be an easy solution for this:

https://github.com/juju/juju/pull/8177 (merged)
https://github.com/juju/juju/pull/8177/files
https://github.com/juju/juju/pull/8188 (pending)
https://github.com/juju/juju/pull/8188/files

It adds several useful mechanisms:

1. cloudinit-userdata model config key

2. a way to execute certain runcmd commands before Juju's own commands and after them

* runcmd key is not allowed in userdata because it would override everything
* preruncmd - before Juju-generated runcmd
* postruncmd - after Juju-generated runcmd

We can simply use cloud-init-provided config options with that:

http://cloudinit.readthedocs.io/en/latest/topics/examples.html#add-apt-repositories
http://cloudinit.readthedocs.io/en/latest/topics/examples.html#additional-apt-configuration

disable_suites: [$RELEASE-security]

or just:

  primary:
...
      uri: http://<local-main>/ubuntu
...
  security:
      uri: http://<local-security>/ubuntu

I think that we can rely on that instead of using apt-mirror now.

The PRs above are mostly about allowing a user to customize userdata. The actual delivery to the provider code path (at least when I look at MAAS) has been there for a while:

https://github.com/juju/juju/blame/juju-2.3.1/provider/maas/environ.go#L851-L887
https://github.com/juju/gomaasapi/blame/master/machine.go#L226-L262
https://github.com/maas/maas/blame/2.3.0/src/maasserver/api/machines.py#L463-L469

Passing non-Juju-generated user-data to containers is also addressed in the above PRs AFAIKs.

[John A Meinel (jameinel)]

Note that any time you actively need to use custom userdata is probably a
time where Juju is failing to model something important. So while it is an
outlet to getting something that works, it should still be a bug that we
don't support the underlying use case correctly.

John
=:->

On Sun, Dec 10, 2017 at 4:15 AM, Dmitrii Shcherbakov <
<email address hidden>> wrote:

> cloud-init user-data customization seems to be an easy solution for
> this:
>
> https://github.com/juju/juju/pull/8177 (merged)
> https://github.com/juju/juju/pull/8177/files
> https://github.com/juju/juju/pull/8188 (pending)
> https://github.com/juju/juju/pull/8188/files
>
> It adds several useful mechanisms:
>
> 1. cloudinit-userdata model config key
>
> 2. a way to execute certain runcmd commands before Juju's own commands
> and after them
>
> * runcmd key is not allowed in userdata because it would override
> everything
> * preruncmd - before Juju-generated runcmd
> * postruncmd - after Juju-generated runcmd
>
> We can simply use cloud-init-provided config options with that:
>
> http://cloudinit.readthedocs.io/en/latest/topics/examples.
> html#add-apt-repositories
> http://cloudinit.readthedocs.io/en/latest/topics/examples.
> html#additional-apt-configuration
>
> disable_suites: [$RELEASE-security]
>
> or just:
>
> primary:
> ...
> uri: http://<local-main>/ubuntu
> ...
> security:
> uri: http://<local-security>/ubuntu
>
>
> I think that we can rely on that instead of using apt-mirror now.
>
> The PRs above are mostly about allowing a user to customize userdata.
> The actual delivery to the provider code path (at least when I look at
> MAAS) has been there for a while:
>
> https://github.com/juju/juju/blame/juju-2.3.1/provider/
> maas/environ.go#L851-L887
> https://github.com/juju/gomaasapi/blame/master/machine.go#L226-L262
> https://github.com/maas/maas/blame/2.3.0/src/maasserver/
> api/machines.py#L463-L469
>
> Passing non-Juju-generated user-data to containers is also addressed in
> the above PRs AFAIKs.
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1599886
>
> Title:
> apt-mirror does not override security.ubuntu.com for containers on
> trusty
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1599886/+subscriptions
>

[Ante Karamatić (ivoks)]

@John I fully agree. All this information is already available to juju, so IMHO juju should just use it. Instead of picking up information from MAAS, juju should just take a peak into host's cloud-config data and use it to generate LXD and KVM cloud-config. Obviously, it should override some settings with its own, but instead of implementing all these features, it would be much easier to consume the already available data. Reading cloud-config from the host also makes it provider agnostic.

[Joseph Phillips (manadart)]

Trusty is no longer a supported Juju series.