`juju upgrade-controller` behaves weirdly with non-existent agent streams

[Jack Shaw]

$ juju version
2.9.22-ubuntu-amd64

I believe this only applies to k8s controllers/models, but it could be more general

Normally, when upgrading a controller or model, Juju searches for new versions to upgrade to in a particular stream (either `released` or `proposed`) found here:
https://streams.canonical.com/juju/tools/streams/v1/

Note: An absent `--agent-stream` flag leads to juju defaulting to released.

Examples:
$ juju status
Model Controller Cloud/Region Version SLA Timestamp
m microk8s-controller microk8s/localhost 2.9.22 unsupported 16:45:49Z

$ juju upgrade-controller --dry-run --agent-stream released
no upgrades available

$ juju upgrade-controller --dry-run --agent-stream proposed
best version:
    2.9.25

All as expected so far
However, in the case that you try a nonsense stream such as:

$ juju upgrade-controller --dry-run --agent-stream blah-blah-blah
best version:
    2.9.26

Note that version 2.9.26 is the latest compatible tag available on dockerhub as of writing this.

This is very useful behavior (particularly in juju CI), but should not be triggered through a nonsense agent-stream. There should be a sensible way to do this

[Ian Booth (wallyworld)]

To summarise the behaviour, juju looks for simplestreams metadata matching the supplied agent-stream. For machine models, this metadata exactly defines what's needed to locate the relevant agent tarball to download and use.

It does seem like there's a bug if upgrade allows you to specify a stream named "foo" or anything else that's not "released", "proposed", "devel", "testing". We'd need to check that we're not relying on this behaviour for CI though where we build up bespoke metadata to pull the binary under test, but in that case we should be using "testing". So still a bug and we should tweak how CI works :-)

For k8s models, we use the streams versions to then filter out the tagged container images - the images are tagged with just version so simplestreams metadata filtering narrows down the available tags to released, proposed etc.

However, if the developer-mode feature flag is on, or the agent-stream is something other than "released", we ignore the simplestreams filter and just select any tagged image.
I think I would argue that if the agent-stream is "proposed", we should still filter on that, so there's a second bug. We don't routinely publish to streams other than "released" or "proposed" so just filtering on those would be all that's needed.
The extra k8s logic is in toolVersionsForCAAS()