charm proxy settings are not generating proxy config file

[Jeff Hillman]

Configuring https_proxy and http_proxy in the containerd charm config options is not creating /etc/systemd/system/containerd.service.d/proxy.conf

This is causing workloads to no deploy and returning an imagePullBackoff

Manually creating that file and putting in the appropriate settings resolves the issue.

This was first noticed in rev 64 in the charm store. Starting a new deploy with rev 61 does not see this issue and the file is created as expected.

[Joseph Borg (joeborg)]

Confirmed this on CK stable. Off hand, I can't see what's changed to cause this.

[Joseph Borg (joeborg)]

I don't think this is due to version, I think this is due to when

```
juju model-config juju-http-proxy=http://squid.internal:3128 juju-https-proxy=http://squid.internal:3128
```

is run. I don't think juju model-config causes a config-changed, so this can be worked around by causing a containerd config changed.

Will look into how to handle this hook.

[Jeff Hillman (jhillman)]

FYI, I am not setting any juju model-level proxy settings either before, during, or after the installation.

The only proxy config I have is in containerd.

[Joseph Borg (joeborg)]

Unfortunately, this seems to be a limitation in Juju as the model config does not cause a config-changed, or call any other hook.

The work around is to set it on the application as well:

```
juju config containerd http_proxy=http://54.89.20.19:3128 https_proxy=http://54.89.20.19:3128
```

This will trigger a config-changed which will write the file.

[Joseph Borg (joeborg)]

Looking deeper, it seems you need to set both. Not sure what's changed here, will investigate.

[Tim Van Steenburgh (tvansteenburgh)]

Subscribing field-medium since the original bug (https://bugs.launchpad.net/charm-docker/+bug/1850862) was.

[Joseph Borg (joeborg)]

PR https://github.com/charmed-kubernetes/charm-containerd/pull/40

[Kevin W Monroe (kwmonroe)]

I have a possible explanation of why this started happening in rev 64. We stopped setting containerd.ready in config_changed() and started setting it in charm_status() here:

https://github.com/charmed-kubernetes/charm-containerd/pull/31/files#diff-48a885c5068a45a19f138133911cd78cL328

Perhaps on initial deployment, install_containerd() runs but 'ctr' isn't quite ready, so we fail to set containerd.ready. Subsequently, an operator changes proxy config; config.changed and config.changed.*_proxy are set, and config_changed() runs.

At this point, containerd.ready still isn't set because we haven't done the @atexit charm_status() yet. This means proxy_changed() won't run because we gate it on the .ready flag:

https://github.com/charmed-kubernetes/charm-containerd/blob/bb37aa56000e67d039f5a46849b2555f67a6b89a/reactive/containerd.py#L346

When the config-changed hook is done, we *do* set .ready, but now the config.changed.*_proxy flags are gone (cleared automatically at the end of each hook invocation):

https://charmsreactive.readthedocs.io/en/latest/layer-basic.html#reactive-flags-for-charm-config

So we won't get into proxy_changed() unless the operator changes proxy config again while the .ready flag is set.

[Joseph Borg (joeborg)]

Ah thanks Kevin, that makes sense.

[George Kraft (cynerva)]

Cherry-pick to stable: https://github.com/charmed-kubernetes/charm-containerd/commit/ab531419164531898152745ae4f426c00a82eff9