juju cli should not fail because of pending environment

[Ahmed Kamal]

Everytime I bootstrap or deploy something .. I always wait an undetermined amount of time for ec2 to "catch-up"
Can we have a "wait" option such that the ensemble command doesn't return till the operation is completed
ensemble bootstrap [-w | --wait]

This should be supported for various operations:
- Bootstrap
- deploy (till it's started, or install_error I guess + I guess should set exit status)
- add-unit

Perhaps there should even be a "ensemble wait" that waits on all async operations to finish, so I can like

for i in `seq 1 5`; do ensemble add-unit foo ; done
ensemble wait

[Clint Byrum (clint-fewbar)]

This is *essential* for writing full end to end integration tests.

I wrote a script which emulates this, but it really should be integrated into ensemble's CLI commands as --wait where it makes sense.

Here is a link to the wait4state script I created:

http://bazaar.launchpad.net/~clint-fewbar/ensemble/jenkins-test-suite/view/head:/misc/jenkins/wait4state

[Gustavo Niemeyer (niemeyer)]

This feels like a workaround for something that would generally be better done in a different way.

[Gustavo Niemeyer (niemeyer)]

For instance, the fact you want to wait for bootstrapping is generally a side-effect of the lack of compound commands (stacks).

[Clint Byrum (clint-fewbar)]

Excerpts from Gustavo Niemeyer's message of Tue Sep 13 18:35:28 UTC 2011:
> For instance, the fact you want to wait for bootstrapping is generally a
> side-effect of the lack of compound commands (stacks).
>

At some point we have to let go of the idea that everybody can write
effective asynchronous scripts. It may not be at the "wait for bootstrap"
level, but users need to be able to write straight forward, blocking
scripts, or they'll do evil things like polling with sub-second timeouts.

Even with a stack, I can see wanting

deploy-stack --wait-for-relations

Which means wait for all relations in the stack to be up before returning.
This would allow somebody's continuous deployment system to *know* that
the new stack at least did everything *it* expected to do already. If
you make them poll *all* of the services, you're not really helping them
with automation. More than likely they'll just pick the one they perceive
as being "last" and poll that.. possibly racing with other services and
then having a less reliable experience when some new formula slows down
that process.

Its a wishlist item, we can live without it. We can workaround it by
doing what I did, polling status, but seems kind of silly to have that
as an external script when it would be *super* easy inside ensemble.

[Juan L. Negron (negronjl)]

I tend to agree with Clint on this one. This would be an awesome thing to
have available.
This could make demos and automation really flow.

Thanks,

Juan

On Tue, Sep 13, 2011 at 12:18 PM, Clint Byrum <email address hidden> wrote:

> Excerpts from Gustavo Niemeyer's message of Tue Sep 13 18:35:28 UTC 2011:
> > For instance, the fact you want to wait for bootstrapping is generally a
> > side-effect of the lack of compound commands (stacks).
> >
>
> At some point we have to let go of the idea that everybody can write
> effective asynchronous scripts. It may not be at the "wait for bootstrap"
> level, but users need to be able to write straight forward, blocking
> scripts, or they'll do evil things like polling with sub-second timeouts.
>
> Even with a stack, I can see wanting
>
> deploy-stack --wait-for-relations
>
> Which means wait for all relations in the stack to be up before returning.
> This would allow somebody's continuous deployment system to *know* that
> the new stack at least did everything *it* expected to do already. If
> you make them poll *all* of the services, you're not really helping them
> with automation. More than likely they'll just pick the one they perceive
> as being "last" and poll that.. possibly racing with other services and
> then having a less reliable experience when some new formula slows down
> that process.
>
> Its a wishlist item, we can live without it. We can workaround it by
> doing what I did, polling status, but seems kind of silly to have that
> as an external script when it would be *super* easy inside ensemble.
>
> --
> You received this bug notification because you are subscribed to
> Ensemble.
> https://bugs.launchpad.net/bugs/849071
>
> Title:
> Ensemble needs to support wait cli option
>
> Status in Ensemble:
> Confirmed
>
> Bug description:
> Everytime I bootstrap or deploy something .. I always wait an undetermined
> amount of time for ec2 to "catch-up"
> Can we have a "wait" option such that the ensemble command doesn't return
> till the operation is completed
> ensemble bootstrap [-w | --wait]
>
> This should be supported for various operations:
> - Bootstrap
> - deploy (till it's started, or install_error I guess + I guess should set
> exit status)
> - add-unit
>
> Perhaps there should even be a "ensemble wait" that waits on all async
> operations to finish, so I can like
>
> for i in `seq 1 5`; do ensemble add-unit foo ; done
> ensemble wait
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ensemble/+bug/849071/+subscriptions
>
>

[Gustavo Niemeyer (niemeyer)]

I didn't say the idea is entirely without merit, but I do think that this should be evaluated on a per case basis, rather than the silver bullet being sold in the description.

Your example is also questionable:

> deploy-stack --wait-for-relations

Please show this being useful in a non-error-prone way. The relation being established for Ensemble doesn't say much about it being ready for the underlying service, and the next action you attempt would have to wait on the _actual_ service to be ready.

So, again, I'm understand the conceptual problem, and I'm happy to work with you guys to improve the situation, but let's try to provide tools that will enable the user to do things reliably.

[Juan L. Negron (negronjl)]

This is actually very good. Gustavo understands the idea. He may not
completely agree with the implementation but the idea is out there and it is
a good one.

Gustavo, I understand your concern about the implementation. Would you
then suggest a more feasible way of implementing or accomplishing this?

Thanks,

Juan

Sent from my mobile.
Apologies for the brevity of this message.
On Sep 13, 2011 1:25 PM, "Gustavo Niemeyer" <email address hidden> wrote:
> I didn't say the idea is entirely without merit, but I do think that
> this should be evaluated on a per case basis, rather than the silver
> bullet being sold in the description.
>
> Your example is also questionable:
>
>> deploy-stack --wait-for-relations
>
> Please show this being useful in a non-error-prone way. The relation
> being established for Ensemble doesn't say much about it being ready for
> the underlying service, and the next action you attempt would have to
> wait on the _actual_ service to be ready.
>
> So, again, I'm understand the conceptual problem, and I'm happy to work
> with you guys to improve the situation, but let's try to provide tools
> that will enable the user to do things reliably.
>
> --
> You received this bug notification because you are subscribed to
> Ensemble.
> https://bugs.launchpad.net/bugs/849071
>
> Title:
> Ensemble needs to support wait cli option
>
> Status in Ensemble:
> Confirmed
>
> Bug description:
> Everytime I bootstrap or deploy something .. I always wait an undetermined
amount of time for ec2 to "catch-up"
> Can we have a "wait" option such that the ensemble command doesn't return
till the operation is completed
> ensemble bootstrap [-w | --wait]
>
> This should be supported for various operations:
> - Bootstrap
> - deploy (till it's started, or install_error I guess + I guess should set
exit status)
> - add-unit
>
> Perhaps there should even be a "ensemble wait" that waits on all async
> operations to finish, so I can like
>
> for i in `seq 1 5`; do ensemble add-unit foo ; done
> ensemble wait
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ensemble/+bug/849071/+subscriptions
>

[Jim Baker (jimbaker)]

Proposed change discussed on the mailing list: https://lists.ubuntu.com/archives/juju/2011-November/000990.html