snap ignores the proxy environment variables

[Jian Luo]

I'm working behind an http proxy and find out that, snappy stopped to using http_proxy and/or https_proxy for connection with store since version 2. The proxy setting works with snappy 1.x, wget and curl.

root@SnapEntw:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04 LTS
Release: 16.04
Codename: xenial

root@SnapEntw:~# uname -a
Linux SnapEntw 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

root@SnapEntw:~# apt-cache show snapd
Package: snapd
Priority: optional
Section: devel
Installed-Size: 18536
Maintainer: Ubuntu Developers <email address hidden>
Architecture: amd64
Version: 2.0.2
Replaces: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9)
Depends: init-system-helpers (>= 1.18~), libc6 (>= 2.4), adduser, lsb-release, squashfs-tools, ubuntu-core-launcher (>= 1.0.23)
Conflicts: snap (<< 2013-11-29-1ubuntu1), snappy
Breaks: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9)
Filename: pool/main/s/snapd/snapd_2.0.2_amd64.deb
Size: 4010516
MD5sum: 45b58056b9107a47e185f4a9069e5044
SHA1: 003a857233ebd9315c4d5e86efea8015f1d0f28a
SHA256: 27507f566e314d3f1f65b609c7007b69b69249d82233bbfbd1d8a507eeda9d24
Description-en: Tool to interact with Ubuntu Core Snappy.
 Manage an Ubuntu system with snappy.
Description-md5: 364ba9499ea9442fb2a40f0fe7925d19
Built-Using: golang-1.6 (= 1.6.1-0ubuntu1), golang-check.v1 (= 0.0+git20150729.11d3bc7-2), golang-defaults (= 2:1.6-1ubuntu4), golang-github-coreos-go-systemd (= 3-2), golang-github-gorilla-mux (= 0.0~git20150814.0.f7b6aaa-1), golang-github-gosexy-gettext (= 0~git20130221-0ubuntu6), golang-github-mvo5-goconfigparser (= 0.2.1-0ubuntu1), golang-github-mvo5-uboot-go (= 0~3.git69978a3-0ubuntu5), golang-github-peterh-liner (= 0.0~git20151118.0.4d47685-1), golang-go-flags (= 0.0~git20160302-0ubuntu1), golang-go.crypto (= 1:0.0~git20151201.0.7b85b09-2), golang-gopkg-tomb.v2 (= 0.0~git20140626.14b3d72-1), golang-pb (= 0.0~git20131219-1), golang-pty (= 0.0~git20150511.1.5cf931e-1ubuntu1), golang-websocket (= 0.0~git20150811.0.b6ab76f-1), golang-yaml.v2 (= 0.0+git20160301.0.a83829b-1)
Homepage: https://github.com/ubuntu-core/snappy
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 5y
Task: ubuntu-core, ubuntu-desktop, ubuntu-usb, cloud-image, server, kubuntu-desktop, ubuntu-core, edubuntu-desktop, edubuntu-usb, xubuntu-core, xubuntu-desktop, mythbuntu-desktop, lubuntu-core, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntu-gnome-desktop, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-mate-cloudtop

root@SnapEntw:~# export http_proxy=http://192.168.1.123:3128/
root@SnapEntw:~# export https_proxy=http://192.168.1.123:3128/

root@SnapEntw:~# snap install hello-world

error: cannot perform the following tasks:
- Download snap "ubuntu-core" from channel "stable" (Get https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin&q=package_name%3Aubuntu-core: dial tcp: lookup search.apps.ubuntu.com on 127.0.1.1:53: no such host)

root@SnapEntw:~# wget https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin&q=package_name%3Aubuntu-core
[1] 2536
root@SnapEntw:~# The name is too long, 268 chars total.
Trying to shorten...
New name is search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Cti.
--2016-05-09 10:02:25-- https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin
Connecting to 192.168.1.123:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 123898 (121K) [application/hal+json]
Saving to: ‘search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Cti.1’

search?fields=anon_download_url%2Cchannel%2Cdownloa 100%[==================================================================================================================>] 120,99K 628KB/s in 0,2s

2016-05-09 10:02:26 (628 KB/s) - ‘search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Cti.1’ saved [123898/123898]

root@SnapEntw:~# curl -v https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin&q=package_name%3Aubuntu-core
[1] 2538
root@SnapEntw:~# * Trying 192.168.1.123...
* Connected to 192.168.1.123 (192.168.1.123) port 3128 (#0)
* Establish HTTP proxy tunnel to search.apps.ubuntu.com:443
> CONNECT search.apps.ubuntu.com:443 HTTP/1.1
> Host: search.apps.ubuntu.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< Connection: close
<
* Proxy replied OK to CONNECT request
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 692 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: search.apps.ubuntu.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: C=GB,ST=Southwark,L=London,O=Canonical Group Ltd,CN=search.apps.ubuntu.com
* start date: Tue, 02 Jun 2015 00:00:00 GMT
* expire date: Mon, 06 Jun 2016 12:00:00 GMT
* issuer: C=US,O=DigiCert Inc,CN=DigiCert SHA2 Secure Server CA
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin HTTP/1.1
> Host: search.apps.ubuntu.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 09 May 2016 08:03:37 GMT
< Server: gunicorn/19.3.0
< X-Suggested-Currency: EUR
< Access-Control-Expose-Headers: X-Suggested-Currency
< Expires: Mon, 09 May 2016 08:04:38 GMT
< Vary: Accept,Accept-Language,Authorization,X-GeoIP-Country-Code,X-Ubuntu-Architecture,X-Ubuntu-Device-Channel,X-Ubuntu-Frameworks,X-Ubuntu-Slots,X-Ubuntu-Release,X-Ubuntu-Store
< X-Bzr-Revision-Number: 461
< Cache-Control: max-age=61
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET
< Content-Type: application/hal+json
< Content-Length: 123898
< X-Cache: MISS from juju-prod-ols-cpi-machine-7
< X-Cache-Lookup: HIT from juju-prod-ols-cpi-machine-7:3128
< Via: 1.1 juju-prod-ols-cpi-machine-7 (squid/3.3.8)
< Strict-Transport-Security: max-age=2592000
<

[Gustavo Niemeyer (niemeyer)]

The "snap" tool is a client to snapd. Setting environment variables on the client won't make snapd aware of them. We need to find a nice way to set such configuration details on the server.

[Gustavo Niemeyer (niemeyer)]

If anyone else stumbles on this issue meanwhile, please note you can set the environment configuration via a systemd configuration file.

Here are some hints about how to do this:

http://serverfault.com/questions/413397/how-to-set-environment-variable-in-systemd-service

[Gustavo Niemeyer (niemeyer)]

John has a more elegant response to this same question here:

http://askubuntu.com/questions/764610/how-to-install-snap-packages-behind-web-proxy-on-ubuntu-16-04/773230#773230

[Jian Luo (luojian)]

Thanks Gustavo. That's well explained.

[Oliver Grawert (ogra)]

on default ubuntu installs /etc/environment only contains the PATH variable (on very old upgraded systems there might be a LANGUAGE/LANG one as well, but that was dropped at some point)

/etc/environment is only writable for the system admin (who would also be the one adding proxy info there) so i think it is safe to just include EnvironmentFile=/etc/environment by default in the snapd service file.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Julian Alarcon (julian-alarcon)]

Reported duplicated bug on Ubuntu package https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1574702

[hackel (hackel)]

So what is the fix? How do you get snaps to pick up environment variables?

[lorenzo (lorenzosu)]

It looks like http_proxy isn't picked by snap.

Because I am using an authenticating proxy I'm not sure I should put that information in a system-wide /etc/environment

Why can't snap just use standard environment variables http_proxy?

[Jason Hobbs (jason-hobbs)]

I got this to work: http://paste.ubuntu.com/p/K8VncJv4vp/

[Gao Shichao (xgdgsc)]

What if I just want to use proxy temporarily and don' t want to mess up with /etc/environment. Why can' t the client send a temporary config to the server?

[Frédéric DANNA (frederic-danna-h)]

@ Jason Hobbs: The solution you gave doesn't work for me:
```
> systemctl show snapd.service | grep -i proxy
DropInPaths=/etc/systemd/system/snapd.service.d/override.conf /etc/systemd/system/snapd.service.d/snap_proxy.conf

> cat /etc/systemd/system/snapd.service.d/snap_proxy.conf
[Service]
Environment=http_proxy=http://frederic.danna:pwd=foo@proxy:8080
Environment=https_proxy=http://frederic.danna:pwd=foo@proxy:8080

>sudo systemctl daemon-reload
>sudo systemctl restart snapd.service

>sudo snap install gitkraken
erreur : cannot install "gitkraken": Post https://api.snapcraft.io/v2/snaps/refresh: Proxy
          Authentication Required
```

I thought my pbm could come from the "=" char in my password, so I also tried with the URL-encoded version of my password, "pwd%3Dfoo". Same error: "Proxy Authentication Required".

It looks like the proxy IP and port are ok, but the proxy authent data (login and/or password) are ignored...

Any idea?

[John Lenton (chipaca)]

I've just checked, and both pwd=foo and pwd%3D=foo in the userinfo part of the url get picked up with no issue (and amount to the same thing).

What is your proxy seeing?

[John Lenton (chipaca)]

er, i meant pwd%3Dfoo, obvs