setting up system proxy via cloud-config might be nice

[Scott Moser]

as seen in bug 1089389 it might be nice to set system wide proxy information. There seems to be reasonable explanation of how you'd do that at [1].
Config could look like:

system_proxy:
 http_proxy: http://foo.bar:3128
 https_proxy: https://foo.host:3128
 ftp_proxy: ...
 no_proxy: ...

cloud-init would/should take just the lower case and write /etc/environment correctly.

Some tricky things here:
 a.) cloud-init should then adjust its own environment to use these settings (and subprocesses)
 b.) what about the datasource? ie, should / would access to http://169.254.169.254 be proxied?

[1] http://askubuntu.com/questions/150210/how-do-i-set-systemwide-proxy-servers-in-xubuntu-lubuntu-or-ubuntu-studio

[Geronimo Orozco (gorozco)]

+1 add repo behind proxy or installing packages is a nightmare

[Dustin Kirkland  (kirkland)]

Please. I need this.

[Jeroen T. Vermeulen (jtv)]

Given the comments, can we raise the priority on this one?

[Giuseppe Iannello (giuseppe-iannello)]

+1
Deploying a machine with a private IP only (EC2/VPC) would benefit from this

[Maitreya Ranganath (maitreya)]

+1

However, I found a workaround to proceed without these properties within cloud-init.
Setting environment variables within /etc/profile or /etc/environment did not work since cloud-init is started by init(8) and does not inherit these variables.
I had to use the config files in /etc/init/cloud-*.conf to set the relevant environment variables before invoking the cloud init scripts.

For example in /etc/init/cloud-config.conf:

# cloud-config - Handle applying the settings specified in cloud-config
description "Handle applying cloud-config"

start on (filesystem and started rsyslog)
console output
task

env HTTP_PROXY=http://x.y.z.w:8080/
env HTTPS_PROXY=http://x.y.z.w:8080/
env NO_PROXY=169.254.169.254

env http_proxy=http://x.y.z.w:8080/
env https_proxy=http://x.y.z.w:8080/
env no_proxy=169.254.169.254

export HTTP_PROXY
export HTTPS_PROXY
export NO_PROXY
export http_proxy
export https_proxy
export no_proxy

exec cloud-init-cfg all config

This set up the environment for all the scripts executed by cloud-init, including curl, wget, apt-get. I was able to successfully test this in AWS EC2 in a purely private Subnet and sending requests to the Internet via my on-prem proxy server. The 'no_proxy' environment variables are to allow the AWS CLI and SDK to retrieve and use data and credentials from the EC2 instance meta-data service.

[Sam Stoelinga (sammiestoel)]

+1 for priority raise

[attaboystrong (attaboystrong)]

Adding our experience based on Maitreya Ranganath (maitreya)'s answer:

The file /etc/init.d/cloud-config had below entries.

# If there exist sysconfig/default variable override files use it...
[ -f /etc/sysconfig/cloud-init ] && . /etc/sysconfig/cloud-init
[ -f /etc/default/cloud-init ] && . /etc/default/cloud-init

so we created /etc/default/cloud-init that contains below proxy variables.
export HTTP_PROXY=http://x.y.z.w:8080/
export HTTPS_PROXY=http://x.y.z.w:8080/
export http_proxy=http://x.y.z.w:8080/
export https_proxy=http://x.y.z.w:8080/

As a result, the file /etc/default/cloud-init is executed when /etc/init.d/cloud-config is executed during boot. This provides access to internet to be able to download stuff required in clout-init process.

[Graham Leggett (minfrin-y)]

In reference to "so we created /etc/default/cloud-init that contains below proxy variables" - how did you do that?

I am assuming you didn't use cloud-init to do it, or did you?

If not, what did you use?

[Peter Schroeter (schroeter)]

+1 for priority raise also, have bumped into this issue a couple times now. I also think this feature would be quite useful as well. VPCs and other locked downed environments are only going to increase in popularity so it'd useful having an easy way of pulling down scripts/installing packages in that scenario via cloud-config.

[Szádeczky-Kardoss Szabolcs (szabolcs-c)]

This was also an issue in our case - but there's a solution.
In case of CentOS 7.1 and yum updates this is one way:

#cloud-config
runcmd:
 - echo "proxy=http://proxy.example.com:8080" >> /etc/yum.conf
 - echo proxy_username=user123 >> /etc/yum.conf
 - echo proxy_password=pwd456 >> /etc/yum.conf
 - echo "http_proxy=http://user123:<email address hidden>:8080" > /etc/profile.d/proxy.sh && chmod 755 /etc/profile.d/proxy.sh
 - echo "https_proxy=http://user123:<email address hidden>:8080" >> /etc/profile.d/proxy.sh
 - echo "no_proxy=localhost,127.0.0.1,169.254.169.254,.sock,.example.com" >> /etc/profile.d/proxy.sh
 - yum update -y
...

[Szádeczky-Kardoss Szabolcs (szabolcs-c)]

Sorry, in my previous script the "export" keyword was missing for http_proxy, https_proxy and no_proxy:
...
 - echo "export http_proxy=http://user123:<email address hidden>:8080" > /etc/profile.d/proxy.sh && chmod 755 /etc/profile.d/proxy.sh
 - echo "export https_proxy=http://user123:<email address hidden>:8080" >> /etc/profile.d/proxy.sh
 - echo "export no_proxy=localhost,127.0.0.1,169.254.169.254,.sock,.example.com" >> /etc/profile.d/proxy.sh
...

[Ben Howard (darkmuggle-deactivatedaccount)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

An alternative might be to use a boot hook.

#cloud-boothook
#!/bin/sh

#### Ubuntu/Debian

# Add proxies to Apt config
cat > /etc/apt/apt.conf.d/00-proxy <<EOC
Acquire::http { Proxy "http://proxy.example.com:8080"; };
Acquire::https { Proxy "http://proxy.example.com:8080"; };
Acquire::http::Pipeline-Depth "23";
Acquire::Languages "none";
EOC

# Add it to the environment
cat >> /etc/environment <<EOC
http_proxy="http://myproxy.server.com:8080/"
https_proxy="http://myproxy.server.com:8080/"
ftp_proxy="http://myproxy.server.com:8080/"
no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"
HTTP_PROXY="http://myproxy.server.com:8080/"
HTTPS_PROXY="http://myproxy.server.com:8080/"
FTP_PROXY="http://myproxy.server.com:8080/"
NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"
EOC

# YUM
cat > /etc/yum.conf <<EOC
proxy=http://proxy.example.com:8080"
echo proxy_username=user123
proxy_password=pwd456
EOC
-----BEGIN PGP SIGNATURE-----

iQIbBAEBCgAGBQJWN3EQAAoJEETWil5UBqhmIhYP9RuvBvysYbP4rXVa01UkWJQq
pQOa3gg1Mp2izTz6mQHCDhOWitNGLy2/bTW1y5mq2XXC/P+EUyy0FLX3YmXObUSa
hnEy6QCkHI1SAwqPCkSOE2dgcWQE3//8AW6xb4uTA+klaEqQAZ5Npa+WqXZlRFkU
+o8EEhExGEUn/E983wmYL6HABMHEVc52RX4THi8WsIkcHkRnoipu1wPnyP0EwBdI
uioMLyEHa7eWRp6BKQN6Gol7ElDMGccKdjxMb2mh0zk9Jo8ckK/IDQZvAAmpbqrB
mmNSiNd7vxE9Qu98I2sxXqQNkMCyLxL1Z6zXwoAQK4MRz1BBEJxckpKJCkVqbNL9
rEgtsl+GOBCEVk8ukyMipH9l+bhPj49tfn+jkHjotI9Dll4ajXF+TuSSpx/1gXAM
/i3uRGiWEzx85ZMSSsYBn+X7XnYtzQgMLMB84+JKGkEwcS/3beZfGQeemrziEHc8
CaBjV2gM6SXJbHr95RNJyULD9o6UzRzt63eSMPfnacYSFuHmCn2k3HMxM6xumwwK
HqiZEKMRRdVupDowzxQ+yLeDZG5nXa4wKDuPFHgFqBRDLgfvoakmuSGoJ3Eb6cde
A4VU3HyZjEN8DBiiF9HVpQXjkjefpnnIGCFkZV3fybY85bLROsKa7AuQveRc0MJ3
eT2lFZkKHQ/qOU200zg=
=SlyK
-----END PGP SIGNATURE-----

[Elson Rodriguez (elson-rodriguez)]

At the root of this issue is the inability to define a global environment variable for cloud-init.

A patchwork of workarounds can be made, but as long as we can't define an environment variable that is inherited by all processes spawned by cloud-init, there will always be an issue waiting to happen.

[Scott Moser (smoser)]

I'll say that I would accept patches for setting up environment for cloud-init.
Something in cloud-config that just did:

environment:
 env_name1: env_name2

and then cloud-init on start up would just read those and basically:
  os.environ.update(cfg.get(environment), {})

or something to that affect.

[DanyChen (danychen)]

I solved it with this:

bootcmd:
- |
  cloud-init-per once env sh -c "mkdir -p /etc/systemd/system/cloud-config.service.d &&
  mkdir -p /etc/systemd/system/cloud-final.service.d && { cat > /etc/cloud/env <<-EOF
  http_proxy=http://192.168.1.10:7890
  https_proxy=http://192.168.1.10:7890
  all_proxy=socks5://192.168.1.10:7891
  EOF
  } && { cat > /etc/systemd/system/cloud-config.service.d/override.conf <<-EOF
  [Service]
  EnvironmentFile=/etc/cloud/env
  EOF
  } && { cat > /etc/systemd/system/cloud-final.service.d/override.conf <<-EOF
  [Service]
  EnvironmentFile=/etc/cloud/env
  EOF
  } && systemctl daemon-reload"

so the proxy server will be in use in cloud-config and cloud-final stage, which most of the operation is done, eg: install package, runcmd.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2335