snap ignores the proxy environment variables

Bug #1579652 reported by Jian Luo
126
This bug affects 28 people
Affects Status Importance Assigned to Milestone
Snappy
Fix Released
Undecided
John Lenton
snapd (Ubuntu)
Fix Released
Undecided
John Lenton

Bug Description

I'm working behind an http proxy and find out that, snappy stopped to using http_proxy and/or https_proxy for connection with store since version 2. The proxy setting works with snappy 1.x, wget and curl.

root@SnapEntw:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04 LTS
Release: 16.04
Codename: xenial

root@SnapEntw:~# uname -a
Linux SnapEntw 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

root@SnapEntw:~# apt-cache show snapd
Package: snapd
Priority: optional
Section: devel
Installed-Size: 18536
Maintainer: Ubuntu Developers <email address hidden>
Architecture: amd64
Version: 2.0.2
Replaces: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9)
Depends: init-system-helpers (>= 1.18~), libc6 (>= 2.4), adduser, lsb-release, squashfs-tools, ubuntu-core-launcher (>= 1.0.23)
Conflicts: snap (<< 2013-11-29-1ubuntu1), snappy
Breaks: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9)
Filename: pool/main/s/snapd/snapd_2.0.2_amd64.deb
Size: 4010516
MD5sum: 45b58056b9107a47e185f4a9069e5044
SHA1: 003a857233ebd9315c4d5e86efea8015f1d0f28a
SHA256: 27507f566e314d3f1f65b609c7007b69b69249d82233bbfbd1d8a507eeda9d24
Description-en: Tool to interact with Ubuntu Core Snappy.
 Manage an Ubuntu system with snappy.
Description-md5: 364ba9499ea9442fb2a40f0fe7925d19
Built-Using: golang-1.6 (= 1.6.1-0ubuntu1), golang-check.v1 (= 0.0+git20150729.11d3bc7-2), golang-defaults (= 2:1.6-1ubuntu4), golang-github-coreos-go-systemd (= 3-2), golang-github-gorilla-mux (= 0.0~git20150814.0.f7b6aaa-1), golang-github-gosexy-gettext (= 0~git20130221-0ubuntu6), golang-github-mvo5-goconfigparser (= 0.2.1-0ubuntu1), golang-github-mvo5-uboot-go (= 0~3.git69978a3-0ubuntu5), golang-github-peterh-liner (= 0.0~git20151118.0.4d47685-1), golang-go-flags (= 0.0~git20160302-0ubuntu1), golang-go.crypto (= 1:0.0~git20151201.0.7b85b09-2), golang-gopkg-tomb.v2 (= 0.0~git20140626.14b3d72-1), golang-pb (= 0.0~git20131219-1), golang-pty (= 0.0~git20150511.1.5cf931e-1ubuntu1), golang-websocket (= 0.0~git20150811.0.b6ab76f-1), golang-yaml.v2 (= 0.0+git20160301.0.a83829b-1)
Homepage: https://github.com/ubuntu-core/snappy
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 5y
Task: ubuntu-core, ubuntu-desktop, ubuntu-usb, cloud-image, server, kubuntu-desktop, ubuntu-core, edubuntu-desktop, edubuntu-usb, xubuntu-core, xubuntu-desktop, mythbuntu-desktop, lubuntu-core, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntu-gnome-desktop, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-mate-cloudtop

root@SnapEntw:~# export http_proxy=http://192.168.1.123:3128/
root@SnapEntw:~# export https_proxy=http://192.168.1.123:3128/

root@SnapEntw:~# snap install hello-world

error: cannot perform the following tasks:
- Download snap "ubuntu-core" from channel "stable" (Get https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin&q=package_name%3Aubuntu-core: dial tcp: lookup search.apps.ubuntu.com on 127.0.1.1:53: no such host)

root@SnapEntw:~# wget https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin&q=package_name%3Aubuntu-core
[1] 2536
root@SnapEntw:~# The name is too long, 268 chars total.
Trying to shorten...
New name is search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Cti.
--2016-05-09 10:02:25-- https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin
Connecting to 192.168.1.123:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 123898 (121K) [application/hal+json]
Saving to: ‘search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Cti.1’

search?fields=anon_download_url%2Cchannel%2Cdownloa 100%[==================================================================================================================>] 120,99K 628KB/s in 0,2s

2016-05-09 10:02:26 (628 KB/s) - ‘search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Cti.1’ saved [123898/123898]

root@SnapEntw:~# curl -v https://search.apps.ubuntu.com/api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin&q=package_name%3Aubuntu-core
[1] 2538
root@SnapEntw:~# * Trying 192.168.1.123...
* Connected to 192.168.1.123 (192.168.1.123) port 3128 (#0)
* Establish HTTP proxy tunnel to search.apps.ubuntu.com:443
> CONNECT search.apps.ubuntu.com:443 HTTP/1.1
> Host: search.apps.ubuntu.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< Connection: close
<
* Proxy replied OK to CONNECT request
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 692 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: search.apps.ubuntu.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: C=GB,ST=Southwark,L=London,O=Canonical Group Ltd,CN=search.apps.ubuntu.com
* start date: Tue, 02 Jun 2015 00:00:00 GMT
* expire date: Mon, 06 Jun 2016 12:00:00 GMT
* issuer: C=US,O=DigiCert Inc,CN=DigiCert SHA2 Secure Server CA
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /api/v1/search?fields=anon_download_url%2Cchannel%2Cdownload_sha512%2Csummary%2Cdescription%2Cbinary_filesize%2Cdownload_url%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Csnap_id%2Csupport_url%2Ctitle%2Ccontent%2Cversion%2Corigin HTTP/1.1
> Host: search.apps.ubuntu.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 09 May 2016 08:03:37 GMT
< Server: gunicorn/19.3.0
< X-Suggested-Currency: EUR
< Access-Control-Expose-Headers: X-Suggested-Currency
< Expires: Mon, 09 May 2016 08:04:38 GMT
< Vary: Accept,Accept-Language,Authorization,X-GeoIP-Country-Code,X-Ubuntu-Architecture,X-Ubuntu-Device-Channel,X-Ubuntu-Frameworks,X-Ubuntu-Slots,X-Ubuntu-Release,X-Ubuntu-Store
< X-Bzr-Revision-Number: 461
< Cache-Control: max-age=61
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET
< Content-Type: application/hal+json
< Content-Length: 123898
< X-Cache: MISS from juju-prod-ols-cpi-machine-7
< X-Cache-Lookup: HIT from juju-prod-ols-cpi-machine-7:3128
< Via: 1.1 juju-prod-ols-cpi-machine-7 (squid/3.3.8)
< Strict-Transport-Security: max-age=2592000
<

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

The "snap" tool is a client to snapd. Setting environment variables on the client won't make snapd aware of them. We need to find a nice way to set such configuration details on the server.

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

If anyone else stumbles on this issue meanwhile, please note you can set the environment configuration via a systemd configuration file.

Here are some hints about how to do this:

http://serverfault.com/questions/413397/how-to-set-environment-variable-in-systemd-service

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :
Revision history for this message
Jian Luo (luojian) wrote :

Thanks Gustavo. That's well explained.

Revision history for this message
Oliver Grawert (ogra) wrote :

on default ubuntu installs /etc/environment only contains the PATH variable (on very old upgraded systems there might be a LANGUAGE/LANG one as well, but that was dropped at some point)

/etc/environment is only writable for the system admin (who would also be the one adding proxy info there) so i think it is safe to just include EnvironmentFile=/etc/environment by default in the snapd service file.

John Lenton (chipaca)
Changed in snappy:
assignee: nobody → John Lenton (chipaca)
status: New → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in snapd (Ubuntu):
status: New → Confirmed
Revision history for this message
Julian Alarcon (julian-alarcon) wrote :

Reported duplicated bug on Ubuntu package https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1574702

John Lenton (chipaca)
Changed in snappy:
status: In Progress → Fix Released
Changed in snapd (Ubuntu):
status: Confirmed → Fix Released
assignee: nobody → John Lenton (chipaca)
Revision history for this message
hackel (hackel) wrote :

So what is the fix? How do you get snaps to pick up environment variables?

Revision history for this message
lorenzo (lorenzosu) wrote :

It looks like http_proxy isn't picked by snap.

Because I am using an authenticating proxy I'm not sure I should put that information in a system-wide /etc/environment

Why can't snap just use standard environment variables http_proxy?

Revision history for this message
Jason Hobbs (jason-hobbs) wrote :
Revision history for this message
Gao Shichao (xgdgsc) wrote :

What if I just want to use proxy temporarily and don' t want to mess up with /etc/environment. Why can' t the client send a temporary config to the server?

Revision history for this message
Frédéric DANNA (frederic-danna-h) wrote :

@ Jason Hobbs: The solution you gave doesn't work for me:
```
> systemctl show snapd.service | grep -i proxy
DropInPaths=/etc/systemd/system/snapd.service.d/override.conf /etc/systemd/system/snapd.service.d/snap_proxy.conf

> cat /etc/systemd/system/snapd.service.d/snap_proxy.conf
[Service]
Environment=http_proxy=http://frederic.danna:pwd=foo@proxy:8080
Environment=https_proxy=http://frederic.danna:pwd=foo@proxy:8080

>sudo systemctl daemon-reload
>sudo systemctl restart snapd.service

>sudo snap install gitkraken
erreur : cannot install "gitkraken": Post https://api.snapcraft.io/v2/snaps/refresh: Proxy
          Authentication Required
```

I thought my pbm could come from the "=" char in my password, so I also tried with the URL-encoded version of my password, "pwd%3Dfoo". Same error: "Proxy Authentication Required".

It looks like the proxy IP and port are ok, but the proxy authent data (login and/or password) are ignored...

Any idea?

Revision history for this message
John Lenton (chipaca) wrote :

I've just checked, and both pwd=foo and pwd%3D=foo in the userinfo part of the url get picked up with no issue (and amount to the same thing).

What is your proxy seeing?

Revision history for this message
John Lenton (chipaca) wrote :

er, i meant pwd%3Dfoo, obvs

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers