juju upgrade-charm errors when it shouldn't

[Tim Penhey]

I have an environment with cs:trusty/postgresql-10 deployed.

I wanted to upgrade it to the latest, which is now cs:trusty/postgresql-28.

Unfortunately Juju said "no".

cs:trusty/postgresql-10 defines the following peer relation:

peers:
  replication:
    interface: pgreplication

and cs:trusty/postgresql-28 has a different interface name (stub didn't realise it would cause a problem).

peers:
  replication:
    interface: pgpeer

I have just one unit of postgresql, and since it is a peer relation, it is automatically related to itself. The charm upgrade code notices that the interface changes, and stops the upgrade.

Juju should allow this upgrade, even if it is only when there is one unit.

[Stuart Bishop (stub)]

ERROR cannot upgrade service "postgresql" to charm "local:trusty/postgresql-1": would break relation "postgresql:replication"

Alas, the interface name cannot be changed back without breaking even more deployments.

People trying to upgrade from the old PostgreSQL charm will need to branch the PostgreSQL charm, edit metadata.yaml changing 'interface: pgpeer' to 'interface: pgreplication', and upgrade to this branch. This will become unnecessary after upgrading to a version of juju-core with the fix.

[Stuart Bishop (stub)]

This just blocked an upgrade on a 1.25.8 environment.

[Stuart Bishop (stub)]

Sorry, Juju version was 1.25.6 if that makes any difference.

[Anastasia (anastasia-macmood)]

@Stuart,

Is there anyway you can try it out with later 1.25.x? I think 1.25.8...

[Richard Harding (rharding)]

I fully expect this to be the same. It's a general rule that you can't
change the interfaces like this that are in use and since it's a peer
relation it's kind of 'always in use' except that when you have one unit
that's not quite true. I think we'll have to update to be intelligent to
treating "a single unit of a peer only relation" as a special case which
hasn't been done to date.

On Tue, Dec 6, 2016 at 10:58 AM Anastasia <email address hidden>
wrote:

> @Stuart,
>
> Is there anyway you can try it out with later 1.25.x? I think 1.25.8...
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> https://bugs.launchpad.net/bugs/1510787
>
> Title:
> juju upgrade-charm errors when it shouldn't
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1510787/+subscriptions
>

[Stuart Bishop (stub)]

Heck, the only problem I'm aware of is the PostgreSQL charm so you could just encode a special rule for it in 1.25.X...

[Anastasia (anastasia-macmood)]

We are not likely to encode "a special rule" just for PostgreSQL charm :)

Since this is the only charm affected and there is a workaround, I am marking this bug as "Won't Fix" for 1.25.x

[Anastasia (anastasia-macmood)]

PR against develop (2.2): https://github.com/juju/juju/pull/7189

[John A Meinel (jameinel)]

I'm curious what a "correct" answer for applications in active peer relations that want to upgrade.

Consider what happens if a charm author wants to clean up how they are doing peer relations, and people have actively deployed the charm with >1 unit. What is the way forward, as they can't break the peer relation, and telling them "juju remove-unit" back down to one really isn't going to cut it if you have something like a 12 node cassandra ring, probably your data won't fit on one node.
If it was "rev 1 has peer A, rev 2 peer A & B, rev 3 peer B" and we could migrate between them, that might be ok, since that gives a time where the peers will be communicating over a different peer.

The other option is that we don't block on peer relations at all, because of the assumption that soon all of them will be on the new version of the charm talking the new peer relation. (I'm leaning toward the latter.)

I don't have a lot of experience with how charms interact over peer relations and what might happen during an upgrade if some disappear from one relation scope to reappear in the other.

[Stuart Bishop (stub)]

Nothing should block charm upgrades that the operator cannot resolve without destroying the service.

For a peer relation, there doesn't seem to be any choice except destroying the removed peer relation.

If that breaks things, it is my responsibility as charm maintainer to make an update that recovers the situation. I can't unrelease charms that have been deployed, but I can certainly make new releases that pick up the pieces. And operators need to be able to apply those new releases, rather than being blocked.

[Anastasia (anastasia-macmood)]

@Stuart Bishop (stub),
We've created bug # 1694956 to track progress on the work that will make sure that we do not block peer relations upgrades.