Upgrade --upload-tools fails if there's a series in streams Juju doesn't recognize

[Nate Finch]

Every six months we run into the same problem - there's a new series announced, and it makes it into part of the ecosystem but not all.

I was doing upgrade testing today and got this error:

ERROR juju.worker runner.go:219 exited "upgrader": invalid series "wily"

This blocked my environment from upgrading. Why is that? I wasn't deploying tools for the series wily... and I wasn't upgrading a machine that used the series wily. So why does my environment care that somewhere someone specified a series it doesn't understand if it's not even using that series?

I think this is overly aggressive error checking when retrieving streams. We definitely should never stop someone's environment from upgrading because someone published something to streams that our juju doesn't recognize. If it doesn't recognize it, it shouldn't use it... but it shouldn't just stop everything from proceeding.

Comment #2 described two work arounds. Many people report that updating the distro-info-data package on the state-server (often machine-0) fixes the issue
     sudo apt-get install distro-info-data

[Curtis Hovey (sinzui)]

This issue overlaps with Tims assertion that the state-server's job is to store the agents, not judge them: bug 1403689

[Curtis Hovey (sinzui)]

There is still a problem for envs on 1.21.x which don't have the upgrade fix. Users can upgrade to a fixed version of juju because 1.22.x has wily agents. User who want to upgrade need to consider these options

A. Use --upload-tools to bypass streams using 1.22.6 or 1.24.2 clients
   juju upgrade-juju --upload-tools
and then restore streams doing
    juju set-env agent-metadata-url=https://streams.canonical.com/juju/tools

B. Manufacture streams without wily (and maybe vivid agents) and publish them
    juju set-env agent-metadata-url=https://<host>/<path>/tools
    juju upgrade-juju --version=<1.22.6|1.24.2>
    juju set-env agent-metadata-url=https://streams.canonical.com/juju/tools

[Andreas Hasenack (ahasenack)]

For the record, (B) worked for me. I just had to remove wily, btw.

[Paul Gear (paulgear)]

Per the pastebin linked in https://bugs.launchpad.net/juju-core/+bug/1498349, option A in comment #2 doesn't work, because "juju upgrade-juju --upload-tools" also fails with the same error.

Can you provide more detail about how to manufacture the streams required by the first step in option B?

[Curtis Hovey (sinzui)]

Hi Paul. Juju QA has test that shows 1.20 reliably upgrades to 1.24. Can you provide some details about the state-server and client so that we can make our test better? Making streams is a pain and I hope we can find a simple work around. For both the state-server and the client we want to know:
1. what are their ubuntu series
2. What are their juju-versions
3. is "wily" in /usr/share/distro-info/ubuntu.csv on localhost and the state-server (provided by distro-info-data)
4. What version was the state-server before 1.20.x?

[Paul Gear (paulgear)]

@chris-gondolin has fixed this environment using the technique described in option B - details at https://pastebin.canonical.com/140580/

[Anastasia (anastasia-macmood)]

targeted against old milestone