Fail on enabling HA

[Marcelo Subtil Marcal]

When bootstrapping a HA controller set using the 2.8.10-focal version, only the first controller gets ready.

Looking at the logs, we could see:
-----
2021-06-09 16:12:21 ERROR juju.packaging.manager utils.go:109 packaging command failed: exit status 1; cmd: "snap install core"; output: error: cannot install "core": persistent network error: Post
       https://api.snapcraft.io/v2/snaps/refresh: dial tcp 91.189.92.20:443:
       connect: network is unreachable
-----

The other two controllers didn't get the proper snap-proxy setting, i.e.

sudo snap get system proxy.http{,s}

Returns empty.

[Marcelo Subtil Marcal (msmarcal)]

Workaround

Set manually the snap proxy variable:

sudo snap set system proxy.http{,s}="PROXY_URL"

[Jeff Hillman (jhillman)]

subscribed field high. this is affect multiple deployments and happens very consistently. not reproduceable on 2.9.4

[Marcelo Subtil Marcal (msmarcal)]

model_defaults.yaml used on bootstrapping the controllers:
-----
apt-http-proxy: http://[REDACTED]:8000
apt-https-proxy: http://[REDACTED]:8000
cloudinit-userdata: "write_files:\n - content: |\n kernel.keys.maxkeys = 2000\n\
  \ owner: \"root:root\"\n path: /etc/sysctl.d/10-maxkeys.conf\n permissions:\
  \ \"0644\"\npostruncmd:\n - sysctl --system\n"
juju-http-proxy: http://[REDACTED]:8000
juju-https-proxy: http://[REDACTED]:8000
juju-no-proxy: [REDACTED]/24,[REDACTED]/24,localhost
snap-http-proxy: http://[REDACTED]:8000
snap-https-proxy: http://[REDACTED]:8000
-----

[Pen Gale (pengale)]

How are you setting the initial snap proxy? Are you passing a --config argument to the bootstrap command?

I'm guessing that this is an issue w/ a bootstrap config value not actually making it into the controller model config, but I wanted to verify.

Also, are the controller machines VMs, or bare metal? Where do they sit, in terms of hardware and networks, relative to each other?

[Marcelo Subtil Marcal (msmarcal)]

Hi Pete,

It's done by FCE.

The foundation.log shows:

2021-06-04-21:57:24 root DEBUG ['juju', 'bootstrap', '--bootstrap-constraints', 'tags=juju', '--config', 'bootstrap-timeout=1800', '--bootstrap-series', 'focal', '--model-default', '/home/ubuntu/[REDACTED]/generated/model_defaults.yaml', 'maas_cloud', 'foundations-maas']

[Marcelo Subtil Marcal (msmarcal)]

The controller machines are VMs sit on infra nodes, and they are on the OAM network.

[John A Meinel (jameinel)]

Since this doesn't appear to still be an issue, I'm unsubscribing Field High. If it is still an issue we can look to address it again.
However, since it appears to already be fixed in 2.9 I'm not sure that the answer is going to be much more that 'please upgrade to 2.9'.