[2.5.2] Juju client does not respect cidr notation in no_proxy variable

Bug #1819993 reported by Dmitrii Shcherbakov
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Triaged
Low
Unassigned

Bug Description

Juju client needs access to the following destinations:

* charm store;
* streams.canonical.com (tools and gui metadata);
* machine provider (for which no_proxy is often configured);
* juju controller.

Specifying something like no_proxy=192.0.2.0/24 in an environment variable doesn't change the client logic to avoid the configured proxy servers for that subnet.

As juju-no-proxy supports this notation (and a similar one for domains) it would be good if the client also did.

As far as I can tell, Juju client code path mainly uses the same code the daemons do so it should work:

https://github.com/juju/juju/blob/juju-2.5.2/cmd/juju/commands/main.go#L182-L188
 if err := proxy.DefaultConfig.Set(proxyutils.DetectProxies()); err != nil {
  return errors.Trace(err)
 }
 if err := proxy.DefaultConfig.InstallInDefaultTransport(); err != nil {

https://github.com/juju/proxy/blob/018dd322e60918d40e166c54821684c05739ef8c/proxy.go#L41-L49

https://github.com/juju/juju/blob/juju-2.5.2/utils/proxy/proxyconfig.go#L124-L136 GetProxy
https://github.com/juju/juju/blob/juju-2.5.2/utils/proxy/proxyconfig.go#L43-L122 useProxy (which parses no_proxy, including cidrs every time a new HTTP connection is to be made to a certain destination)

However, I am getting an error from macaroon-bakery.v2-unstable/httpbakery/client.go so it might be somehow unaffected by InstallInDefaultTransport:

https_proxy=http://192.0.2.20:8000
http_proxy=http://192.0.2.20:8000
no_proxy=localhost,127.0.0.1,192.0.2.0/23,infra-1,infra-2,infra-3,.maas

01:32:36 DEBUG httpbakery client.go:243 client do POST https://192.0.2.135:17070/model/3f6e5cc1-5bc5-4d20-8a29-3e1a4d8cc662/charms?revision=0&schema=local&series=bionic {
01:32:36 DEBUG httpbakery client.go:245 } -> error [{/build/juju/parts/juju/go/src/github.com/juju/juju/vendor/gopkg.in/macaroon-bakery.v2-unstable/httpbakery/client.go:273: } {Post https://192.0.2.135:17070/model/3f6e5cc1-5bc5-4d20-8a29-3e1a4d8cc662/charms?revision=0&schema=local&series=bionic: Forbidden}]
01:32:36 DEBUG juju.api monitor.go:35 RPC connection died
ERROR cannot deploy bundle: POST https://192.0.2.135:17070/model/3f6e5cc1-5bc5-4d20-8a29-3e1a4d8cc662/charms?revision=0&schema=local&series=bionic: Post https://192.0.2.135:17070/model/3f6e5cc1-5bc5-4d20-8a29-3e1a4d8cc662/charms?revision=0&schema=local&series=bionic: Forbidden
01:32:36 DEBUG cmd supercommand.go:496 error stack:

Changed in juju:
status: New → Triaged
importance: Undecided → High
milestone: none → 2.6-beta1
Changed in juju:
milestone: 2.6-beta1 → 2.6-beta2
Changed in juju:
milestone: 2.6-beta2 → 2.6-rc1
Changed in juju:
milestone: 2.6-rc1 → 2.6-rc2
Changed in juju:
milestone: 2.6-rc2 → 2.6.1
Changed in juju:
milestone: 2.6.1 → 2.6.2
Changed in juju:
milestone: 2.6.2 → 2.6.3
Changed in juju:
milestone: 2.6.3 → 2.6.4
Changed in juju:
assignee: nobody → Achilleas Anagnostopoulos (achilleasa)
Changed in juju:
status: Triaged → In Progress
Revision history for this message
Achilleas Anagnostopoulos (achilleasa) wrote :

Based on the attached log output I am assuming that you are trying to deploy
a local charm. That would trigger the POST operation that seems to be failing.

I am trying to replicate this issue but with no luck so far. Here is what I
tried so far:

1) Install tinyproxy and set it up in forward mode:
$ apt-get install -y tinyproxy

$ cat > /tmp/tinyproxy.conf <<EOT
Port 8081
# NOTE: this is the IP on my local machine
Listen 192.168.0.127

LogLevel Connect
PidFile "/tmp/tinyproxy.pid"
LogFile "/tmp/tinyproxy.log"
MaxClients 100
MinSpareServers 2
MaxSpareServers 5
StartServers 2
MaxRequestsPerChild 0
EOT

$ tinyproxy -d -c /tmp/tinyproxy.conf

2) Bootstrap a test controller using juju 2.5.2. I have tried both compiling
from the source and the published centos build (https://launchpad.net/juju/2.5/2.5.2/+download/juju-2.5.2-centos7.tar.gz).

3) Export proxy settings. My lxd instances receive IPs in subnet 10.65.47.0/24.

$ export http_proxy=192.168.0.127:8081
$ export https_proxy=192.168.0.127:8081
$ export no_proxy=localhost,127.0.0.1,10.65.47.0/24

4) Try a local bundle deployment while tailing /tmp/tinyproxy.log after
exporting the proxy settings:

$ juju deploy $GOPATH/src/github.com/juju/juju/testcharms/charm-repo/quantal/dummy --debug

This call does not hit the proxy for the POST to the charm upload endpoint. If
I remove the 10.65.47.0/24 subnet from "no_proxy" then I can see the POST going
through the proxy as expected.

----

Is this still an issue for you? If so, can you please provide some additional
information (e.g the commands you were trying to run and additional logs) to
help me replicate the issue?

Changed in juju:
status: In Progress → Incomplete
Changed in juju:
milestone: 2.6.4 → 2.6.5
Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

Hi Achilleas,

Yes, it still is.

juju version
2.6.4-bionic-amd64

snap info juju | grep -P 'refresh-date|installed:'
refresh-date: 7 days ago, at 05:48 MSK
installed: 2.6.4 (8241) 68MB classic

juju bootstrap localhost

sudo apt install squid

cd /tmp
charm pull cs:ubuntu

# juju status is fine

https_proxy=http://localhost:3128 http_proxy=http://localhost:3128 no_proxy=localhost,127.0.0.1,10.10.10.0/24 juju status
Model Controller Cloud/Region Version SLA Timestamp
default localhost-localhost localhost/localhost 2.6.4 unsupported 09:30:06+03:00

Model "admin/default" is empty.

With --debug: https://paste.ubuntu.com/p/bqzfV73ZT2/

# juju deploy <local-charm> isn't

https_proxy=http://localhost:3128 http_proxy=http://localhost:3128 no_proxy=localhost,127.0.0.1,10.10.10.0/24 juju deploy /tmp/ubuntu
ERROR POST https://10.10.10.116:17070/model/370e991c-706e-4744-8ada-fc9f06245913/charms?revision=0&schema=local&series=bionic: Post https://10.10.10.116:17070/model/370e991c-706e-4744-8ada-fc9f06245913/charms?revision=0&schema=local&series=bionic: Forbidden

With --debug: https://paste.ubuntu.com/p/JRxsTgKw23/

squid log:

1561703767.286 0 127.0.0.1 TCP_DENIED/403 3948 CONNECT [fdde:b242:ba2:0:216:3eff:fe40:e96a]:17070 - HIER_NONE/- text/html
1561703767.336 0 127.0.0.1 TCP_DENIED/403 3894 CONNECT [fdde:b242:ba2::740]:17070 - HIER_NONE/- text/html
1561703767.520 0 127.0.0.1 TCP_DENIED/403 3858 CONNECT 10.10.10.116:17070 - HIER_NONE/- text/html

`juju deploy` from the store is fine:

https_proxy=http://localhost:3128 http_proxy=http://localhost:3128 no_proxy=localhost,127.0.0.1,10.10.10.0/24 juju deploy cs:ubuntu --debug

https://paste.ubuntu.com/p/MyTPNxqrvH/

Changed in juju:
status: Incomplete → New
Changed in juju:
milestone: 2.6.5 → 2.6.6
Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

For reference, here is a similar bug for snaps which was fixed some time ago. It is also about macaroon-bakery:

https://bugs.launchpad.net/snapd/+bug/1791587

https://bugs.launchpad.net/snapd/+bug/1791587/comments/9

https://github.com/snapcore/snapd/pull/6062

Changed in juju:
milestone: 2.6.6 → 2.6.7
Changed in juju:
assignee: Achilleas Anagnostopoulos (achilleasa) → nobody
milestone: 2.6.7 → 2.7-beta1
John A Meinel (jameinel)
Changed in juju:
status: New → Triaged
Changed in juju:
milestone: 2.7-beta1 → 2.7-rc1
Changed in juju:
milestone: 2.7-rc1 → none
Revision history for this message
Canonical Juju QA Bot (juju-qa-bot) wrote :

This bug has not been updated in 2 years, so we're marking it Low importance. If you believe this is incorrect, please update the importance.

Changed in juju:
importance: High → Low
tags: added: expirebugs-bot
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.