Unable to upgrade hosted model with --upload-tools after upgrading controller with --upload-tools

[Cheryl Jennings]

I upgraded my admin model from 2.0-beta3 to tip of master with --upload-tools, and it completed successfully:

$ juju status -m amazon:admin
[Services]
NAME STATUS EXPOSED CHARM
ubuntu unknown false cs:trusty/ubuntu-7
ubuntu1 unknown false cs:trusty/ubuntu-7

[Units]
ID WORKLOAD-STATUS JUJU-STATUS VERSION MACHINE PORTS PUBLIC-ADDRESS MESSAGE
ubuntu/0 unknown idle 2.0-beta4.1 1 52.37.116.156
ubuntu1/0 unknown idle 2.0-beta4.1 2 52.38.0.77

[Machines]
ID STATE DNS INS-ID SERIES AZ
0 started 52.38.80.2 i-9b390c5c xenial us-west-2a
1 started 52.37.116.156 i-74027aac trusty us-west-2b
2 started 52.38.0.77 i-1d2525c7 trusty us-west-2c

However, when I went to upgrade my hosted model (default), I got the following error:
2016-03-30 18:46:47 ERROR cmd supercommand.go:448 a hosted model cannot have a higher version than the server model: 2.0-beta4.2 > 2.0-beta4.1

[Cheryl Jennings (cherylj)]

Bug #1567170 tracks the removal of --upload-tools for upgrading hosted environments. In future releases, we should allow for users to upgrade a hosted environment to the level of the controller model.

[Curtis Hovey (sinzui)]

Attached is the log of machine-0. While status claims the controller is 2.0.2.1, the agent is no on the controller. The upgrade may have been attempted, but it broken, possibly because of the zesty issue.

     $/var/lib/juju/tools$ ls -l
     drwxr-xr-x 2 root root 4096 Oct 13 18:23 2.0.0.1-xenial-amd64
     drwxr-xr-x 2 root root 4096 Oct 14 18:25 2.0.0-xenial-amd64
     drwxr-xr-x 2 root root 4096 Nov 4 14:35 2.0.1-xenial-amd64
     lrwxrwxrwx 1 root root 38 Nov 4 14:35 machine-0 -> /var/lib/juju/tools/2.0.1-xenial-amd64

[Anastasia (anastasia-macmood)]

Since --upload-tools has been removed in 2.1, I think this bug may need to become "upgrading hosted model/controller bootstrapped with --upload-tools" initially.

@Curtis,

Please confirm :)
I think the case here becomes whether controllers/hosted models that were bootstrapped with Juju 2.0.x (specifically, 2.0.2 and prior) with --upload-tools options do upgrade without an issue.
Please note that 2.0 tip does not have --upload-tools either.

[Launchpad Janitor (janitor)]

[Expired for juju because there has been no activity for 60 days.]

[Tim Penhey (thumper)]

The short answer, is once the controller is updated, you should then upgrade the hosted models specifying the version that the controller now has.

For example if, after controller upgrade, the controller has version 2.0.2.1, then upgrade the hosted models by specifying that agent version.

juju upgrade-juju -m hosted-model --agent-version=2.0.2.1

The hosted models download their agents from the controller, so by specifying the version that the controller has, they will all end up on the same version.