Use proxy from /etc/environment or other system-wide config file

[Martin Pitt]

In the Canonical DC one needs to use a proxy to download images from outside, such as images.linuxcontainers.org. This currently causes some trouble (like in https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/a/autopkgtest/20160314_110014@/log.gz).

AFAIUI, lxd-images is being deprecated, in favor of adding and launching from remotes. The lxd-images command used the proxy information from the current shell, but with "lxc launch images:..." it is the running lxd daemon which does the download, and thus the proxy info on the current shell is meaningless. But the lxd daemon process does not have any proxy environment set.

So I'm looking for a sensible way to configure a default proxy for image downloads. In my local test I just added "EnvironmentFile=-/etc/environment" to /lib/systemd/system/lxd.service's [Service], as that's the most likely place where this is set by default (and that's also where ubuntu-system-service writes it when you configure a system-wide proxy in the control center). Would that be acceptable to add by default?

/etc/environment is normally read by pam_env and thus is mostly meant for user sessions, cron jobs etc.¸ but there's some precedent for sourcing /etc/environment in init.d scripts and upstart jobs.

Alternatively, is there some way that this can be configured in a different file, so that it applies to lxd as soon as it gets apt-get install'ed (via a test dependency)?

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: lxd 2.0.0~rc3-0ubuntu1
ProcVersionSignature: Ubuntu 4.4.0-12.28-generic 4.4.4
Uname: Linux 4.4.0-12-generic x86_64
ApportVersion: 2.20-0ubuntu3
Architecture: amd64
CurrentDesktop: i3
Date: Mon Mar 14 21:18:33 2016
EcryptfsInUse: Yes
SourcePackage: lxd
UpgradeStatus: No upgrade log present (probably fresh install)

[Stéphane Graber (stgraber)]

rc3 introduces new config options for that.

See https://linuxcontainers.org/lxd/news/ or the package changelog for rc3 :)

[Martin Pitt (pitti)]

I actually did notice that, but that doesn't really help much yet -- I know I can set these variables with some "lxc config blah" commands somehow, but how can this happen with dropping config file/snippets somewhere? I normally can't shove a command in between a test installing lxd and running things with it.

The announcement also talks about upper-case environment variables -- I suppose/hope that's just an editorial glitch; but it seems the binary knows about both somehow:

$ strings /usr/bin/lxd|grep -i _proxy
HTTPS_PROXY
HTTP_PROXY
NO_PROXY
http_proxy
https_proxy
no_proxy

[Stéphane Graber (stgraber)]

Yeah, the fallback uses the env variables and checks for both all-uppercase and all-lowercase because apparently some people use the all-capitals syntax...

[Tycho Andersen (tycho-s)]

On Mon, Mar 14, 2016 at 09:10:42PM -0000, Martin Pitt wrote:
> I actually did notice that, but that doesn't really help much yet -- I
> know I can set these variables with some "lxc config blah" commands
> somehow, but how can this happen with dropping config file/snippets
> somewhere? I normally can't shove a command in between a test installing
> lxd and running things with it.
>
> The announcement also talks about upper-case environment variables -- I
> suppose/hope that's just an editorial glitch; but it seems the binary
> knows about both somehow:

Yep, the binary should look at both.

> $ strings /usr/bin/lxd|grep -i _proxy
> HTTPS_PROXY
> HTTP_PROXY
> NO_PROXY
> http_proxy
> https_proxy
> no_proxy
>
> --
> You received this bug notification because you are a member of Ubuntu
> containers team, which is subscribed to lxd in Ubuntu.
> Matching subscriptions: lxd
> https://bugs.launchpad.net/bugs/1557161
>
> Title:
> Use proxy from /etc/environment or other system-wide config file
>
> Status in lxd package in Ubuntu:
> Fix Released
>
> Bug description:
> In the Canonical DC one needs to use a proxy to download images from
> outside, such as images.linuxcontainers.org. This currently causes
> some trouble (like in
> https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac
> /autopkgtest-
> xenial/xenial/amd64/a/autopkgtest/20160314_110014@/log.gz).
>
> AFAIUI, lxd-images is being deprecated, in favor of adding and
> launching from remotes. The lxd-images command used the proxy
> information from the current shell, but with "lxc launch images:..."
> it is the running lxd daemon which does the download, and thus the
> proxy info on the current shell is meaningless. But the lxd daemon
> process does not have any proxy environment set.
>
> So I'm looking for a sensible way to configure a default proxy for
> image downloads. In my local test I just added
> "EnvironmentFile=/etc/environment" to
> /lib/systemd/system/lxd.service's [Service], as that's the most likely
> place where this is set by default (and that's also where ubuntu-
> system-service writes it when you configure a system-wide proxy in the
> control center). Would that be acceptable to add by default?
>
> /etc/environment is normally read by pam_env and thus is mostly meant
> for user sessions, cron jobs etc.¸ but there's some precedent for
> sourcing /etc/environment in init.d scripts and upstart jobs.
>
> Alternatively, is there some way that this can be configured in a
> different file, so that it applies to lxd as soon as it gets apt-get
> install'ed (via a test dependency)?
>
> ProblemType: Bug
> DistroRelease: Ubuntu 16.04
> Package: lxd 2.0.0~rc3-0ubuntu1
> ProcVersionSignature: Ubuntu 4.4.0-12.28-generic 4.4.4
> Uname: Linux 4.4.0-12-generic x86_64
> ApportVersion: 2.20-0ubuntu3
> Architecture: amd64
> CurrentDesktop: i3
> Date: Mon Mar 14 21:18:33 2016
> EcryptfsInUse: Yes
> SourcePackage: lxd
> UpgradeStatus: No upgrade log present (probably fresh install)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/...

[Martin Pitt (pitti)]

I could pre-install /etc/systemd/system/lxc.service.d/proxy.conf with the EnvironmentFile= setting, which gives me what I need. I just wonder if this might not be better to add by default, as you can't really set any env variables otherwise.

[Stéphane Graber (stgraber)]

I'm fine with it if that's a normal systemd thing to do.

[Martin Pitt (pitti)]

Using EnvironmentFile= for slurping in /etc/environment is a standard thing indeed (I just fixed the description to add the missing "-" prefix, in case the file does not exist); this is the counterpart of sourcing /etc/environment into init.d scripts or upstart jobs.

This is more of a philosophical question, whether we want /etc/environment to influence lxd's environment or not.