charm upgrade from charm store can leave units out-of-date and unfixable

[William Reade]

The "upgrade-charm" command does (among other things) the following.

[1]

Abort if the latest charm id matches the *service*'s charm id. This only happens with charm store charms; local charms will have their version incremented, and will be republished with a new id.

[2]

Mark for upgrade only those units currently in a "started" state. In addition, any unit which detects an upgrade flag while not in a "started" state will discard the instruction.

So, consider the situation in which we have some-service/0 (started) and some-service/1 (start_error, say), running cs:precise/some-service-100, and the user upgrades to cs:precise/some-service-101. some-service/0 will detect and process the change, while some-service/1 will ignore it completely; and once you're in this state you'll be prevented from issuing another upgrade to cs:precise/some-service-101, because the service will already be "running" that charm id. I can't see any way to reset the upgrade flag for some-service/1, and I can't upgrade to cs:precise/some-service-102 because that doesn't exist yet; and even if I did, some unit could still discard the instruction.

The problem also exists for local charm repositories, but has a lower impact because it does have a workaround: that is, you can just manually upgrade as many times as it takes for the upgrade to be processed by every machine.

I'm not sure exactly what we should do about this. It seems a touch lumpen to demand that some agent repeatedly reset upgrade flags for units whose charms don't match their service's, but on the other hand I can't actually think of anything better.

[Gustavo Niemeyer (niemeyer)]

> Mark for upgrade only those units currently in a "started" state. In addition, any
> unit which detects an upgrade flag while not in a "started" state will discard the instruction.

Please let me know if I'm going completely off track, but as far as I could perceive in the
description, this is actually the only bug being reported. There's no reason for the unit to
_discard_ the instruction. The upgrade flag is a persistent node in the tree. Once it leaves
the error state and goes back to a normal running state, it should see if there's an upgrade
flag up and proceed as usual.

> I'm not sure exactly what we should do about this. It seems a touch lumpen to demand that
> some agent repeatedly reset upgrade flags for units whose charms don't match their
> service's, but on the other hand I can't actually think of anything better.

We can't do that. The upgrade flag design is precisely addressing the need to upgrade units
individually and at one's discretion.

[Clint Byrum (clint-fewbar)]

Seems to me that this is addressed by not allowing 'upgrade-charm' when any units are in an error state, which is what juju does now. Can somebody confirm that fact? If that's so, then this bug is Invalid.

[Kapil Thangavelu (hazmat)]

its still valid. while the upgrade isn't acted upon in a non error state,
the flag itself is cleared too early, and the watch won't fire again with
the flag value unchanged.

On Fri, Jun 22, 2012 at 2:17 PM, Clint Byrum <email address hidden> wrote:

> Seems to me that this is addressed by not allowing 'upgrade-charm' when
> any units are in an error state, which is what juju does now. Can
> somebody confirm that fact? If that's so, then this bug is Invalid.
>
> ** Changed in: juju
> Milestone: galapagos => honolulu
>
> --
> You received this bug notification because you are subscribed to juju.
> https://bugs.launchpad.net/bugs/917405
>
> Title:
> charm upgrade from charm store can leave units out-of-date and
> unfixable
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/917405/+subscriptions
>

[Clint Byrum (clint-fewbar)]

Ok this seems like a real problem, and easily fixable, so marking as Triaged. Lets try to get this addressed in 0.7.