the ceilometer-upgrade post-deploy action should not be required

[Jason Hobbs]

It's an anti-pattern to require an action (with no parameters) to complete setup after a charm deploy. juju deployed application should handle all of that on their own.

In the case of ceilometer-upgrade, there are "many inherent race conditions" [1] (apparently unresolved at this point) that motivated the addition of this action. It makes it possible for this to work without race conditions, but it is still a hack.

This bug is to track the fact that the relation shouldn't be required. It doesn't provide anything useful except to workaround the unfixed race conditions.

Requiring the charm complicates deploying openstack with juju. It goes from:

juju deploy bundle.yaml

to:

juju deploy bundle.yaml
juju wait --exclude ceilometer
juju run-action ceilometer/leader ceilometer upgrade
juju wait

This is a big complication, given that juju wait is a community supported plugin.

[1] https://github.com/openstack/charm-ceilometer/commit/f3148b9bd74bb08b840dcb45e5ce17736dcab981

[Ryan Beisner (1chb1n)]

The heat charm also has a post-deployment action which has been required for quite some time. With Juju goal state now being a feature, we can revisit this bug, and other areas where ultimate state was not predictable from an individual charm application perspective. In the mean-time, this is the established method to address this type of issue where very serious and impactful race conditions would otherwise exist.

Doing that would require that we take a deep assessment of application state on the whole, and incorporate goal state into the charms. To-date, only a PoC of goal state has been exercised in relation to OpenStack Charms.

Hence, the classification as a feature request.

[Ryan Beisner (1chb1n)]

I forgot to add initially, on the topic of juju-wait:

OpenStack CI test gates rely heavily on juju-wait. Juju-wait is the only known and reliable way to systematically detect when a deployment is "settled" (ie. ready to test, or ready to proceed with post-deployment configuration, or ready for use).

IMO, juju-wait having a community status should be a separate debate from this bug.

[Jason Hobbs (jason-hobbs)]

I respectfully disagree that this is a feature request; it is not enabling
any new functionality.

The fact that the deployment doesn't complete from deploying/relating the
applications is a bug. The action is a workaround. The bug is still there.

The fact that it's a hard bug to fix and that another bug uses the same
workaround doesn't make it less of a bug.

On Mon, Apr 8, 2019 at 11:11 AM Ryan Beisner <email address hidden>
wrote:

> The heat charm also has a post-deployment action which has been required
> for quite some time. With Juju goal state now being a feature, we can
> revisit this bug, and other areas where ultimate state was not
> predictable from an individual charm application perspective. In the
> mean-time, this is the established method to address this type of issue
> where very serious and impactful race conditions would otherwise exist.
>
> Doing that would require that we take a deep assessment of application
> state on the whole, and incorporate goal state into the charms. To-
> date, only a PoC of goal state has been exercised in relation to
> OpenStack Charms.
>
> Hence, the classification as a feature request.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1822190
>
> Title:
> the ceilometer-upgrade post-deploy action should not be required
>
> Status in OpenStack ceilometer charm:
> New
>
> Bug description:
> It's an anti-pattern to require an action (with no parameters) to
> complete setup after a charm deploy. juju deployed application should
> handle all of that on their own.
>
> In the case of ceilometer-upgrade, there are "many inherent race
> conditions" [1] (apparently unresolved at this point) that motivated
> the addition of this action. It makes it possible for this to work
> without race conditions, but it is still a hack.
>
> This bug is to track the fact that the relation shouldn't be required.
> It doesn't provide anything useful except to workaround the unfixed
> race conditions.
>
> Requiring the charm complicates deploying openstack with juju. It goes
> from:
>
> juju deploy bundle.yaml
>
> to:
>
> juju deploy bundle.yaml
> juju wait --exclude ceilometer
> juju run-action ceilometer/leader ceilometer upgrade
> juju wait
>
> This is a big complication, given that juju wait is a community
> supported plugin.
>
> [1] https://github.com/openstack/charm-
> ceilometer/commit/f3148b9bd74bb08b840dcb45e5ce17736dcab981
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/charm-ceilometer/+bug/1822190/+subscriptions
>

[Ryan Beisner (1chb1n)]

The fix is to leverage a feature in Juju which did not previously exist: goal state. To-date, goal state has not been widely adopted or developed within charms. From that perspective, I'm holding the line that it is charm feature work.

[Jason Hobbs (jason-hobbs)]

From a user perspective, goal state isn't providing any new functionality
here. There are no new use cases. It's just an implementation detail that's
part of a fix for something that's broken.

On Tue, Apr 9, 2019 at 11:15 AM Ryan Beisner <email address hidden>
wrote:

> The fix is to leverage a feature in Juju which did not previously exist:
> goal state. To-date, goal state has not been widely adopted or
> developed within charms. From that perspective, I'm holding the line
> that it is charm feature work.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1822190
>
> Title:
> the ceilometer-upgrade post-deploy action should not be required
>
> Status in OpenStack ceilometer charm:
> New
>
> Bug description:
> It's an anti-pattern to require an action (with no parameters) to
> complete setup after a charm deploy. juju deployed application should
> handle all of that on their own.
>
> In the case of ceilometer-upgrade, there are "many inherent race
> conditions" [1] (apparently unresolved at this point) that motivated
> the addition of this action. It makes it possible for this to work
> without race conditions, but it is still a hack.
>
> This bug is to track the fact that the relation shouldn't be required.
> It doesn't provide anything useful except to workaround the unfixed
> race conditions.
>
> Requiring the charm complicates deploying openstack with juju. It goes
> from:
>
> juju deploy bundle.yaml
>
> to:
>
> juju deploy bundle.yaml
> juju wait --exclude ceilometer
> juju run-action ceilometer/leader ceilometer upgrade
> juju wait
>
> This is a big complication, given that juju wait is a community
> supported plugin.
>
> [1] https://github.com/openstack/charm-
> ceilometer/commit/f3148b9bd74bb08b840dcb45e5ce17736dcab981
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/charm-ceilometer/+bug/1822190/+subscriptions
>

[Ryan Beisner (1chb1n)]

Maybe we are talking past one another.

Wishlist != Unimportant

This is important. We will solve it with the modern set of primitives which are designed to address exactly this type of issue, which did not exist until recently.

[James Page (james-page)]

Marking this bug as Triaged.

Yes its a bug and yes its awkward but it does provide a viable solution today so marking this as a Low priority.

Goal state allows us to develop a more tasteful solution to this challenge but its a super heavy lift as for ceilometer to know when it can run ceilometer-upgrade, the entire application stack below it needs to be goal-state aware to cascade ready-ness up application stack. Same applies to heat. and <insert another service here>.

Addressing this as a isolated charm fix won't work; we need to have this as a cross charm goal to enable use of goal state during a project cycle to be able to fix this effectively.

[Alex Kavanagh (ajkavanagh)]

I'm marking this as wont fix as it's now been 4 years and the use of the "ceilometer-upgrade" action is now pretty well embedded in all the tools and 'accepted' as a post-deploy action, along with vault's post-deploy actions and octavia's diskimage-retrofit, etc. Happy to reconsider, but in the scheme of things to work on, this one feels low priority and probably won't float to the top of the pile.