MAAS not respecting proxy

[jarred wilson]

I am using debian package maas/focal 1:2.9.2-9164-g.ac176b5c4-0ubuntu1~20.04.1 all

I am attempting to enlist and commission physical nodes in MAAS that are UCSM power type in the MAAS UI.

MAAS fails commissioning with a "No rack controllers can access the BMC of node <node>" I have added the CA for UCS and am able to curl successfully to the UCS endpoint. Also, I can pass the authentication information with curl to the UCS endpoint and authenticate successfully. Looking at the MAAS logs, it appears to timeout on the UCS login.

This environment currently requires routing through a proxy to get to the UCS endpoint. The proxy is set in the /etc/environment file found here:

PATH="/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/
games:/usr/local/games"
LC_ALL="en_US.UTF-8"
LANG="en_US.UTF-8"
LANGUAGE="en_US:en"
EDITOR="vim"
HTTP_PROXY=http://10.16.63.XXX:8080
http_proxy=http://10.16.63.XXX:8080
HTTPS_PROXY=http://10.16.63.XXX:8080
https_proxy=http://10.16.63.XXX:8080

No_proxy does not include the UCS endpoint. Here is the no_proxy variable set in /etc/profile.d/noproxy.sh:

printf -v no_proxy '%s,' 10.145.{252..2XX}.{1..XXX};
export no_proxy="${no_proxy%,},localhost,::1,127.0.0.1,.infra.domain.net";
export NO_PROXY="${no_proxy%,},localhost,::1,127.0.0.1,.infra.domain.net";

Looking at the tcpdump, MAAS is not respecting the proxy and is trying to directly connect to the UCS endpoint which is being firewalled off. Here is a tcpdump sample:

13:14:13.926729 IP (tos 0x0, ttl 64, id 44899, offset 0, flags [DF], proto TCP (6), length 60)
    10.145.252.XXX.60484 > 10.16.237.XXX.443: Flags [S], cksum 0xfea5 (incorrect -> 0x56fe), seq 1841205033, win 62720, options [mss 8960,sackOK,TS val 4233165060 ecr 0,nop,wscale 7], length 0
13:14:15.942719 IP (tos 0x0, ttl 64, id 44900, offset 0, flags [DF], proto TCP (6), length 60)
    10.145.252.XXX.60484 > 10.16.237.XXX.443: Flags [S], cksum 0xfea5 (incorrect -> 0x4f1e), seq 1841205033, win 62720, options [mss 8960,sackOK,TS val 4233167076 ecr 0,nop,wscale 7], length 0
13:14:20.166717 IP (tos 0x0, ttl 64, id 44901, offset 0, flags [DF], proto TCP (6), length 60)
    10.145.252.XXX.60484 > 10.16.237.XXX.443: Flags [S], cksum 0xfea5 (incorrect -> 0x3e9e), seq 1841205033, win 62720, options [mss 8960,sackOK,TS val 4233171300 ecr 0,nop,wscale 7], length 0

MAAS uses urllib.request to create the requests for use with UCS. From the docs it looks like this should be handled automatically by being pulled from the environmental variables:

In addition, if proxy settings are detected (for example, when a *_proxy environment variable like http_proxy is set), ProxyHandler is default installed and makes sure the requests are handled through the proxy.

Here are the rackd logs: https://pastebin.canonical.com/p/dPtxrF4B9R/

[jarred wilson (jardon)]

subscribed field-critical

[Christian Grabowski (cgrabowski)]

Hi there, I don't believe /etc/environment is applied to systemd units by default (the deb package runs as systemd units), have you tried editing the maas-rackd.service unit and adding the proxy env var to the its environment field, or setting environment-file=/etc/environment?

[jarred wilson (jardon)]

I did add the following to the maas-rackd service files on all three infra nodes and it seems that has resolved the issue:

[Service]
EnvironmentFile=/etc/environment