Unit agents won't start on 2.2.6 model upgraded to 2.3.0

[William Grant]

juju bootstrap --agent-version=2.2.6
juju add-model foo
juju deploy cs:ubuntu
juju deploy cs:telegraf
juju add-relation ubuntu telegraf
juju upgrade-juju -m controller
# Notice bug #1737058 and decide to upgrade unit agents to see if it's a compat issue.
juju upgrade-juju
# Where are my unit agents

unit-ubuntu-0: 06:49:36 DEBUG juju.worker.dependency "uniter" manifold worker stopped: failed to initialize uniter for "unit-ubuntu-0": cannot create relations: cannot get status: relation not found
unit-ubuntu-0: 06:49:36 ERROR juju.worker.dependency "uniter" manifold worker returned unexpected error: failed to initialize uniter for "unit-ubuntu-0": cannot create relations: cannot get status: relation not found

[Andrew Wilkins (axwalk)]

Relation status was introduced in https://github.com/juju/juju/commit/fe60b166756826fc982762c1dee84e272a252483, as part of the CMR work.

In State.Relation.SetStatus, there's an assumption that a relation already has a status: https://github.com/juju/juju/blob/2.3/state/relation.go#L125. No upgrade step was written to set a status for existing relations.

[Andrew Wilkins (axwalk)]

The only workaround I can think of at the moment is to perform Mongo surgery, adding a document to the statuses collection for each relation with the status "joined".

e.g. I have a relation with ID 0 in a model with UUID fa3a2330-bc2d-4dd4-8891-892ad2d992d0. I ran:

    db.statuses.insert({ "_id" : "fa3a2330-bc2d-4dd4-8891-892ad2d992d0:r#0", "model-uuid" : "fa3a2330-bc2d-4dd4-8891-892ad2d992d0", "status" : "joined", "statusinfo" : "joined", "statusdata" : { }, "updated" : NumberLong("1512721579642612830"), "neverset" : true, "txn-revno" : NumberLong(1), "txn-queue" : [ ] })

After running this command, the unit agent continued on its way.

[Andrew Wilkins (axwalk)]

https://github.com/juju/juju/pull/8194

[Tom Haddon (mthaddon)]

This is marked fix released, but https://launchpad.net/~juju/+archive/ubuntu/stable seems to still have the originally released 2.3.0-0ubuntu1 packages. Will those be updated soon?

[Nicholas Skaggs (nskaggs)]

Tom, you just beat the binaries finishing build. The ppa is updated.