juju should support an apt proxy or alternate mirror for private clouds

Bug #897645 reported by Kapil Thangavelu
90
This bug affects 17 people
Affects Status Importance Assigned to Milestone
The Eilt project
Undecided
Unassigned
cloud-init
Medium
Unassigned
juju-core
High
Unassigned
pyjuju
High
Unassigned

Bug Description

private clouds / data centers may not allow public internet access, being able to configure an environment property defining a apt mirror/proxy for instances in the cloud would alleviate this. We should be able to put this into place via cloud-init.

Changed in juju:
milestone: none → florence
status: New → Confirmed
Revision history for this message
Scott Moser (smoser) wrote :

I do not think that juju is the correct place to "support an apt proxy".
Just as cloud-init selects a mirror correctly, JuJu should let cloud-init do apt proxy configuration (if applicable).

The most generic solution for this is to allow users to provide user-data parts to cloud-init.

Then, to support apt proxy selection, the user (with no specific juju support) could add a boothook like:
#!/bin/sh
echo 'Acquire { HTTP { Proxy "http://my-local-proxy:3128/"; }; };' > /etc/apt/apt.conf.d/local-proxy

I also just opened cloud-init bug 897688, which will make specification of apt_proxy supported via cloud-config.

Revision history for this message
Kapil Thangavelu (hazmat) wrote :

Hi Scott,

The goal is to support this via cloud-init, with juju just passing cloud init the right options. ie. we want to use cloud-init support for this functionality, and just populate user-data appropriately if the option is specified on the juju environment. Juju doesn't expose cloud-init to end users, so it has to take care of configuring it as needed.

Revision history for this message
Scott Moser (smoser) wrote :

I'm suggesting that JuJu should allow the end user to specify cloud-init "parts" or a single multi-part file that would then be passed through either prior to the juju generated part or post it (that would have to be thought through) .

This would provide a generic method for local customizations that are going to be necessary.

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

As we discussed, this should be a first class option, for a few reasons:

1) It's simpler. The interface for proxy selection should be:

        juju bootstrap --http-proxy=$URL

2) We want control over that URL.

    Even though we'll be using cloud-init for the moment, we'll also want to enable the user
    to *change* that URL with a command like "juju set-env http-proxy=$URL", and that should
    affect both the future machines, and the *current* machines, which cloud-init does not
    do.

With those said, at some point we may want to enable a generic mechanism for preparing
machines, but this bug isn't justification for it.

Revision history for this message
Scott Moser (smoser) wrote :

As of trunk at revision 486 [1], you can now have cloud-config like:
   apt_proxy: http://my.apt.proxy:3128

That will do what you'd expect.
[1] http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/486

Changed in cloud-init:
status: New → Fix Committed
importance: Undecided → Medium
Revision history for this message
Jorge (correajl) wrote :

It will be very import to Juju allow users to specify some cloud_configs, as we can do creating instances with --user-data-file.

summary: - juju should support an apt proxy for private clouds
+ juju should support an apt proxy or alternate mirror for private clouds
Changed in juju:
importance: Undecided → High
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Note that the case of ongoing-configurability can be handled fine with a subordinate "apt-proxy" charm that would be related to all services. So one way to get a lightweight implementation for this would just be to push apt_proxy: into cloud-init and then have an included apt-proxy subordinate charm that automatically gets related and pushed onto every node.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Just to add my two cents-- I only just started playing with juju for its MIR review, but I'm betting others would want to test drive it too. As such, the LXC option is free of cost (and hassles if you don't have an amazon account already). Many enterprises will have a local mirror or apt proxy and having these options in juju would greatly speed up this trial period. I have to admit that I thought that juju was broken because it took so long for my units to come out of state pending. While this could be handled in documentation (though it would have to be everywhere-- and that is hard to control), for people with access to a local mirror or apt proxy, this would be a great speed-up and improve first impressions. I imagine it would also speed up live demonstrations.

Changed in juju:
milestone: florence → galapagos
Scott Moser (smoser)
Changed in cloud-init:
status: Fix Committed → Fix Released
Changed in juju:
assignee: nobody → Kapil Thangavelu (hazmat)
status: Confirmed → In Progress
Changed in juju:
milestone: galapagos → honolulu
Changed in juju:
status: In Progress → Confirmed
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Kapil, propose merge now or forever hold your peace. I intend to drop this from the 0.6 milestone if its not in review by tomorrow morning.

Changed in juju:
milestone: 0.6 → 0.7
Martin Packman (gz)
Changed in juju-core:
importance: Undecided → High
status: New → Confirmed
Changed in juju:
assignee: Kapil Thangavelu (hazmat) → nobody
Changed in juju:
milestone: 0.7 → 0.8
Revision history for this message
Данило Шеган (danilo) wrote :

This is not very high on the roadmap for juju-core.

Changed in juju-core:
importance: High → Medium
status: Confirmed → Triaged
Revision history for this message
Sidnei da Silva (sidnei) wrote :

Looking at https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/974509 it seems like if the private cloud has a 'ubuntu-mirror' dns name, then cloud-init might pick that up as the mirror.

More specifically, for local provider with lxc, a ubuntu-mirror entry could be pushed to dnsmasq on the host such that it is picked up automatically.

Revision history for this message
Sidnei da Silva (sidnei) wrote :

In other news, while we could pick up the proxy from the host during cloud-init setup, it seems like cloud-init doesn't support setting an https proxy?

Curtis Hovey (sinzui)
Changed in juju-core:
importance: Medium → Low
Revision history for this message
Kapil Thangavelu (hazmat) wrote : Re: [Bug 897645] Re: juju should support an apt proxy or alternate mirror for private clouds

fwiw this is a typical requirement in private clouds.

On Fri, Oct 11, 2013 at 7:52 PM, Curtis Hovey <email address hidden> wrote:

> ** Changed in: juju-core
> Importance: Medium => Low
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/897645
>
> Title:
> juju should support an apt proxy or alternate mirror for private
> clouds
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-init/+bug/897645/+subscriptions
>

Curtis Hovey (sinzui)
Changed in juju:
status: Confirmed → Triaged
Revision history for this message
Curtis Hovey (sinzui) wrote :

I think this feature needs to be solved for 14.04.

tags: added: feature production
Changed in juju-core:
importance: Low → High
Revision history for this message
Jonathan Davies (jpds) wrote :

For private clouds, this can be done in MAAS and MAAS settings has all the options for this.

Curtis Hovey (sinzui)
tags: added: cts-cloud-review
Changed in juju-core:
status: Triaged → Fix Released
milestone: none → 1.17.0
Revision history for this message
Kapil Thangavelu (hazmat) wrote :

how does one enable this behavior?

On Tue, Nov 12, 2013 at 7:16 PM, Dave Cheney <email address hidden>wrote:

> ** Changed in: juju-core
> Status: Triaged => Fix Released
>
> ** Changed in: juju-core
> Milestone: None => 1.17.0
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/897645
>
> Title:
> juju should support an apt proxy or alternate mirror for private
> clouds
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-init/+bug/897645/+subscriptions
>

Revision history for this message
Dave Cheney (dave-cheney) wrote :

If the machine you are bootstrapping on uses the apt proxy, we
magically sniff those settings and pass them to the environment via
cloud init.

On Thu, Nov 14, 2013 at 2:23 AM, Kapil Thangavelu
<email address hidden> wrote:
> how does one enable this behavior?
>
>
> On Tue, Nov 12, 2013 at 7:16 PM, Dave Cheney <email address hidden>wrote:
>
>> ** Changed in: juju-core
>> Status: Triaged => Fix Released
>>
>> ** Changed in: juju-core
>> Milestone: None => 1.17.0
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/897645
>>
>> Title:
>> juju should support an apt proxy or alternate mirror for private
>> clouds
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/cloud-init/+bug/897645/+subscriptions
>>
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> Matching subscriptions: MOAR JUJU SPAM!
> https://bugs.launchpad.net/bugs/897645
>
> Title:
> juju should support an apt proxy or alternate mirror for private
> clouds
>
> Status in Init scripts for use on cloud images:
> Fix Released
> Status in The Eilt project:
> New
> Status in pyjuju:
> Triaged
> Status in juju-core:
> Fix Released
>
> Bug description:
> private clouds / data centers may not allow public internet access,
> being able to configure an environment property defining a apt
> mirror/proxy for instances in the cloud would alleviate this. We
> should be able to put this into place via cloud-init.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-init/+bug/897645/+subscriptions

Revision history for this message
Paul Gear (paulgear) wrote :

The apt-*-proxy settings allow juju to work in egress-restricted environments, but it still does not pick up the mirror settings from the local host. In the parts of the world where we pay for bandwidth outside of our ISP's network, this is a very important setting to have.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers