Formula install hook never completes

[Adam Gandelman]

Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble fires the install hook but its exit seems to not be caught by ensemble *IF* it needs to install the package rabbitmq-server. ensemble status shows its state as 'null' On the machine, the hook is no longer running and log msgs generated by the hook show it is exiting successfully. ensemble.executor does not seem to catch the child process exiting and does not continue on to fire the 'start' hook:

2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate: execute action do_install
2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook: /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook install.
2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
<......snip......>
2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation complete. 0
2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install complete. Exiting 0

However, if I destroy the service and deploy once again (without removing the packages), it successfully installs and moves onto starting:

2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate: transition install (None -> installed) {}
2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate: execute action do_install
2011-07-14 23:57:47,462: hook.executor@DEBUG: Running hook: /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
2011-07-14 23:57:47,803: unit.hook.api@INFO: rabbitmq: Firing hook install.
<<...snip....>
2011-07-14 23:57:49,111: unit.hook.api@INFO: rabbitmq: Installation complete. 0
2011-07-14 23:57:49,511: unit.hook.api@INFO: rabbitmq: Hook install complete. Exiting 0
2011-07-14 23:57:49,554: hook.output@DEBUG: hook install ended, exit code 0.
2011-07-14 23:57:49,554: hook.executor@DEBUG: Hook complete: /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
2011-07-14 23:57:49,562: statemachine@DEBUG: unitworkflowstate: transition complete install (state installed) {}
2011-07-14 23:57:49,563: statemachine@DEBUG: unitworkflowstate: transition state (installed -> started)
2011-07-14 23:57:49,564: statemachine@DEBUG: unitworkflowstate: transition start (installed -> started) {}

If I destroy the service, remove the package and re-deploy, the problem reproduces. From what I can tell, the rabbitmq-server package is fine as it installs fine on a fresh machine, returns appropriate return codes and does not require any debconf/user dialog.

[Thomas Herve (therve)]

I've noticed this problem with 0.5+bzr275-0ensemble1~natty1. Basically if deploy-service or add-unit need to start a new machine, the unit is never usable and the state stays at "null".

[Mark Mims (mark-mims)]

Try the following:

    --- hooks/rabbitmq-relations 2011-07-16 00:57:16 +0000
    +++ hooks/rabbitmq-relations 2011-07-17 21:07:52 +0000
    @@ -22,7 +22,7 @@
       fi

       [[ ! `which pwgen` ]] && apt-get -y install pwgen
    - apt-get -y install rabbitmq-server
    + DEBIAN_FRONTEND=noninteractive apt-get -qqy install --no-install-recommends rabbitmq-server
       rc=$?
       ensemble-log "rabbitmq: Installation complete. $rc"
     }

[Benjamin Saller (bcsaller)]

Because we know the platform we could insert this into the environment
of install hooks on a normal basis if it will make things smoother for
formula developers

On Sun, Jul 17, 2011 at 2:52 PM, Mark Mims <email address hidden> wrote:
> Try the following:
>
>    --- hooks/rabbitmq-relations    2011-07-16 00:57:16 +0000
>    +++ hooks/rabbitmq-relations    2011-07-17 21:07:52 +0000
>    @@ -22,7 +22,7 @@
>       fi
>
>       [[ ! `which pwgen` ]] && apt-get -y install pwgen
>    -  apt-get -y install rabbitmq-server
>    +  DEBIAN_FRONTEND=noninteractive apt-get -qqy install --no-install-recommends rabbitmq-server
>       rc=$?
>       ensemble-log "rabbitmq: Installation complete. $rc"
>     }
>
> --
> You received this bug notification because you are a member of Ensemble
> Composers, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/810808
>
> Title:
>  Formula install hook never completes
>
> Status in Ensemble:
>  New
>
> Bug description:
>  Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble
>  fires the install hook but its exit seems to not be caught by ensemble
>  *IF* it needs to install the package rabbitmq-server.  ensemble status
>  shows its state as 'null'   On the machine, the hook is no longer
>  running and log msgs generated by the hook show it is exiting
>  successfully.   ensemble.executor does not seem to catch the child
>  process exiting and does not continue on to fire the 'start' hook:
>
>  2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate:  execute action do_install
>  2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook: /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
>  2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook install.
>  2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
>  <......snip......>
>  2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation complete. 0
>  2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install complete. Exiting 0
>
>  However, if I destroy the service and deploy once again (without
>  removing the packages), it successfully installs and moves onto
>  starting:
>
>  2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate: transition install (None -> installed) {}
>  2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:  execute action do_install
>  2011-07-14 23:57:47,462: hook.executor@DEBUG: Running hook: /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
>  2011-07-14 23:57:47,803: unit.hook.api@INFO: rabbitmq: Firing hook install.
>  <<...snip....>
>  2011-07-14 23:57:49,111: unit.hook.api@INFO: rabbitmq: Installation complete. 0
>  2011-07-14 23:57:49,511: unit.hook.api@INFO: rabbitmq: Hook install complete. Exiting 0
>  2011-07-14 23:57:49,554: hook.output@DEBUG: hook install ended, exit code 0.
>  2011-07-14 23:57:49,554: hook.executor@DEBUG: Hook complete: /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
>  2011-07-14 23:57:49,562: statemachine@DEBUG: unitworkflowstate: transition complete install (state installed) {}
>  2011-07-14 23:57:49,563: statemachine@DEBUG: unitworkflowstate: transition state (installed...

[Mark Mims (mark-mims)]

Great idea, we should do that... I added an atomic bug 812343 for that.

I think what's happening is that a hook calls a process that's not properly daemonized (has shared process groups and controlling terminals) and hangs the hook. The ensemble reported status is then different from the actual unit status.

Doesn't upstart/sysvinit have similar problems taming/jailing scripts? What do they do to fix it?

It'd be great if we could make ensemble more robust against this.

constraints:
 - we use lots of _long_ running hooks... they update/install/pull/compile/etc
 - anything people can call from a hook... they will
 - ?

[Thomas Herve (therve)]

I don't think that bug is related to what's done during the install hook, fwiw. It happened to me with an empty install script.

[Adam Gandelman (gandelman-a)]

With the proposed patch to the formula, still no sucecss on either current natty or oneiric AMIs.

I'm wondering, is there anyway to increase verbosity of the debug logging or to use any of the tests shipped with ensemble to poke at this further?

[Clint Byrum (clint-fewbar)]

Excerpts from Adam Gandelman's message of Mon Jul 18 17:40:37 UTC 2011:
> With the proposed patch to the formula, still no sucecss on either
> current natty or oneiric AMIs.
>
> I'm wondering, is there anyway to increase verbosity of the debug
> logging or to use any of the tests shipped with ensemble to poke at this
> further?
>

Does it work if you run it manually with debug-hooks ?

Also, maybe you could strace the machine agent, something like

pid=12345
strace -f -o agent-strace.txt -p $pid

You need to do this before the agent is installed though, that may be
tricky. You'd have to deploy an empty formula (no hooks), then destroy
that service, then ssh to the box and start the strace above.

[Adam Gandelman (gandelman-a)]

Adding 'service rabbitmq-server stop' to the end of the install hook seems to cleanup whatever processes might have been blocking ensemble from continuing with deployment.

While I hope ensemble ends up being able to deal with situations like this better in the long term, I'm marking as "invalid" as I believe this bug is related to packaging.

[Thomas Herve (therve)]

I think there is a "bug" in HookProtocol here. If the hooks start a process which is not super nice (I suspect erlang here), stdin and stdout might be forwarded and not closed appropriately. HookProtocol should use processExited to end the hook, not processEnded (which will wait for the pipes to close cleanly).

[Jim Baker (jimbaker)]

That's very interesting and flips the robustness around. processEnded is
used here precisely to wait for stdout/stderr to clean up so that certain
tests can deterministically determine that they the logs are ready to be
ready and verified. So using processEnded currently makes that testing
procedure robust (we have been using Twisted trial's -u looping to have some
more insight into this). Of course the test hook scripts never do the sort
of actions mentioned in Thomas' email.

Incidentally this was changed in r258, and is described a little bit in the
corresponding branch
https://code.launchpad.net/~jimbaker/ensemble/standardize-log-testing

Probably the best step here is to have HookProtocol to support both
processExited and processEnded (presumably same behavior in terms of exit
code not being 0 is an errback, otherwise callback with 0), and making those
deferreds available. Then restore the behavior prior to r258:
Invoker.__call__ will return the deferred associated with processExited.
Lastly, add another method on the Invoker objects so as to be able to wait
on I/O, so that any log verification tests can still work.

On Tue, Jul 19, 2011 at 2:01 AM, Thomas Herve <email address hidden>wrote:

> I think there is a "bug" in HookProtocol here. If the hooks start a
> process which is not super nice (I suspect erlang here), stdin and
> stdout might be forwarded and not closed appropriately. HookProtocol
> should use processExited to end the hook, not processEnded (which will
> wait for the pipes to close cleanly).
>
> --
> You received this bug notification because you are a member of Ensemble
> Composers, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/810808
>
> Title:
> Formula install hook never completes
>
> Status in Ensemble:
> Invalid
>
> Bug description:
> Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble
> fires the install hook but its exit seems to not be caught by ensemble
> *IF* it needs to install the package rabbitmq-server. ensemble status
> shows its state as 'null' On the machine, the hook is no longer
> running and log msgs generated by the hook show it is exiting
> successfully. ensemble.executor does not seem to catch the child
> process exiting and does not continue on to fire the 'start' hook:
>
> 2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate: execute
> action do_install
> 2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook:
> /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
> 2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook
> install.
> 2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
> <......snip......>
> 2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation
> complete. 0
> 2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install
> complete. Exiting 0
>
> However, if I destroy the service and deploy once again (without
> removing the packages), it successfully installs and moves onto
> starting:
>
> 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> transition install (None -> installed) {}
> 2011-07-14 23:57:47,461: statemachine@DEBU...

[Adam Gandelman (gandelman-a)]

Still running into the same issue anytime the hook needs to start the service. I can confirm that replacing processEnded with processExited in HookProtocol seems to get things moving during deploy.

[Jim Baker (jimbaker)]

This is good to know, so it's just a matter of making that change, while
preserving the testing already in place. I think the plan I outlined should
do that.

On Wed, Jul 20, 2011 at 3:17 PM, Adam Gandelman
<email address hidden>wrote:

> Still running into the same issue anytime the hook needs to start the
> service. I can confirm that replacing processEnded with processExited
> in HookProtocol seems to get things moving during deploy.
>
> --
> You received this bug notification because you are a member of Ensemble
> Composers, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/810808
>
> Title:
> Formula install hook never completes
>
> Status in Ensemble:
> In Progress
>
> Bug description:
> Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble
> fires the install hook but its exit seems to not be caught by ensemble
> *IF* it needs to install the package rabbitmq-server. ensemble status
> shows its state as 'null' On the machine, the hook is no longer
> running and log msgs generated by the hook show it is exiting
> successfully. ensemble.executor does not seem to catch the child
> process exiting and does not continue on to fire the 'start' hook:
>
> 2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate: execute
> action do_install
> 2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook:
> /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
> 2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook
> install.
> 2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
> <......snip......>
> 2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation
> complete. 0
> 2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install
> complete. Exiting 0
>
> However, if I destroy the service and deploy once again (without
> removing the packages), it successfully installs and moves onto
> starting:
>
> 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> transition install (None -> installed) {}
> 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate: execute
> action do_install
> 2011-07-14 23:57:47,462: hook.executor@DEBUG: Running hook:
> /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
> 2011-07-14 23:57:47,803: unit.hook.api@INFO: rabbitmq: Firing hook
> install.
> <<...snip....>
> 2011-07-14 23:57:49,111: unit.hook.api@INFO: rabbitmq: Installation
> complete. 0
> 2011-07-14 23:57:49,511: unit.hook.api@INFO: rabbitmq: Hook install
> complete. Exiting 0
> 2011-07-14 23:57:49,554: hook.output@DEBUG: hook install ended, exit code
> 0.
> 2011-07-14 23:57:49,554: hook.executor@DEBUG: Hook complete:
> /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
> 2011-07-14 23:57:49,562: statemachine@DEBUG: unitworkflowstate:
> transition complete install (state installed) {}
> 2011-07-14 23:57:49,563: statemachine@DEBUG: unitworkflowstate:
> transition state (installed -> started)
> 2011-07-14 23:57:49,564: statemachine@DEBUG: unitworkflowstate:
> transition start (installed -> started) {}
>
> If I destroy the service, remove the package and re-deploy, the
> problem reproduces. From ...

[Juan L. Negron (negronjl)]

Hi guys:

This is also blocking the SI Team on the delivery of the cloudfoundry-server formula.
We are experiencing the same issue.
Any chance this can be prioritized high enough to get it fixed as soon as possible?

Thanks,

Juan

[Jim Baker (jimbaker)]

It's definitely high, I just need to finalize the docs on the corresponding
branch to clearly explain the behavior of what's being seen and why the
solution works, per the review discussion.

On Wed, Jul 27, 2011 at 3:46 PM, Juan L. Negron
<email address hidden>wrote:

> Hi guys:
>
> This is also blocking the SI Team on the delivery of the
> cloudfoundry-server formula.
> We are experiencing the same issue.
> Any chance this can be prioritized high enough to get it fixed as soon as
> possible?
>
> Thanks,
>
> Juan
>
> --
> You received this bug notification because you are a member of Ensemble
> Composers, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/810808
>
> Title:
> Formula install hook never completes
>
> Status in CloudFoundry:
> Confirmed
> Status in Ensemble:
> In Progress
>
> Bug description:
> Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble
> fires the install hook but its exit seems to not be caught by ensemble
> *IF* it needs to install the package rabbitmq-server. ensemble status
> shows its state as 'null' On the machine, the hook is no longer
> running and log msgs generated by the hook show it is exiting
> successfully. ensemble.executor does not seem to catch the child
> process exiting and does not continue on to fire the 'start' hook:
>
> 2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate: execute
> action do_install
> 2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook:
> /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
> 2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook
> install.
> 2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
> <......snip......>
> 2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation
> complete. 0
> 2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install
> complete. Exiting 0
>
> However, if I destroy the service and deploy once again (without
> removing the packages), it successfully installs and moves onto
> starting:
>
> 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> transition install (None -> installed) {}
> 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate: execute
> action do_install
> 2011-07-14 23:57:47,462: hook.executor@DEBUG: Running hook:
> /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
> 2011-07-14 23:57:47,803: unit.hook.api@INFO: rabbitmq: Firing hook
> install.
> <<...snip....>
> 2011-07-14 23:57:49,111: unit.hook.api@INFO: rabbitmq: Installation
> complete. 0
> 2011-07-14 23:57:49,511: unit.hook.api@INFO: rabbitmq: Hook install
> complete. Exiting 0
> 2011-07-14 23:57:49,554: hook.output@DEBUG: hook install ended, exit code
> 0.
> 2011-07-14 23:57:49,554: hook.executor@DEBUG: Hook complete:
> /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
> 2011-07-14 23:57:49,562: statemachine@DEBUG: unitworkflowstate:
> transition complete install (state installed) {}
> 2011-07-14 23:57:49,563: statemachine@DEBUG: unitworkflowstate:
> transition state (installed -> started)
> 2011-07-14 23:57:49,564: statemachine@DEBUG: unitworkflowstate:
> transition start (instal...

[Adam Gandelman (gandelman-a)]

Jim's branch fixes the issue well, you can use it in the mean time by specifying ensemble-branch in environments.yml.

[Juan L. Negron (negronjl)]

Thanks Jim for your help on this.
Thanks Adam for pointing out Jim's branch.

Jim, What's the ETA on relase for this?

Thanks,

Juan Negron <email address hidden>
System Integration Engineer
Corporate Services
Canonical USA
Mobile: [phone]
GPG : 0A62 BC70 5CBC B4DD F3E6 8A27 A6B1 F3F0 E6B5 F5A3

On Wed, Jul 27, 2011 at 4:32 PM, Jim Baker <email address hidden>wrote:

> It's definitely high, I just need to finalize the docs on the corresponding
> branch to clearly explain the behavior of what's being seen and why the
> solution works, per the review discussion.
>
> On Wed, Jul 27, 2011 at 3:46 PM, Juan L. Negron
> <email address hidden>wrote:
>
> > Hi guys:
> >
> > This is also blocking the SI Team on the delivery of the
> > cloudfoundry-server formula.
> > We are experiencing the same issue.
> > Any chance this can be prioritized high enough to get it fixed as soon as
> > possible?
> >
> > Thanks,
> >
> > Juan
> >
> > --
> > You received this bug notification because you are a member of Ensemble
> > Composers, which is subscribed to the bug report.
> > https://bugs.launchpad.net/bugs/810808
> >
> > Title:
> > Formula install hook never completes
> >
> > Status in CloudFoundry:
> > Confirmed
> > Status in Ensemble:
> > In Progress
> >
> > Bug description:
> > Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble
> > fires the install hook but its exit seems to not be caught by ensemble
> > *IF* it needs to install the package rabbitmq-server. ensemble status
> > shows its state as 'null' On the machine, the hook is no longer
> > running and log msgs generated by the hook show it is exiting
> > successfully. ensemble.executor does not seem to catch the child
> > process exiting and does not continue on to fire the 'start' hook:
> >
> > 2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate:
> execute
> > action do_install
> > 2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook:
> > /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
> > 2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook
> > install.
> > 2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
> > <......snip......>
> > 2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation
> > complete. 0
> > 2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install
> > complete. Exiting 0
> >
> > However, if I destroy the service and deploy once again (without
> > removing the packages), it successfully installs and moves onto
> > starting:
> >
> > 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> > transition install (None -> installed) {}
> > 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> execute
> > action do_install
> > 2011-07-14 23:57:47,462: hook.executor@DEBUG: Running hook:
> > /var/lib/ensemble/units/rabbitmq-16/formula/hooks/install
> > 2011-07-14 23:57:47,803: unit.hook.api@INFO: rabbitmq: Firing hook
> > install.
> > <<...snip....>
> > 2011-07-14 23:57:49,111: unit.hook.api@INFO: rabbitmq: Installation
> > complete. 0
> > 2011-07-14 23:57:49,511: unit.hook.api@INFO: rabbitmq: Hook install
> > complete....

[Jim Baker (jimbaker)]

I just pushed this branch into trunk, so now :)

On Wed, Jul 27, 2011 at 5:59 PM, Juan L. Negron
<email address hidden>wrote:

> Thanks Jim for your help on this.
> Thanks Adam for pointing out Jim's branch.
>
> Jim, What's the ETA on relase for this?
>
> Thanks,
>
> Juan Negron <email address hidden>
> System Integration Engineer
> Corporate Services
> Canonical USA
> Mobile: [phone]
> GPG : 0A62 BC70 5CBC B4DD F3E6 8A27 A6B1 F3F0 E6B5 F5A3
>
>
> On Wed, Jul 27, 2011 at 4:32 PM, Jim Baker
> <email address hidden>wrote:
>
> > It's definitely high, I just need to finalize the docs on the
> corresponding
> > branch to clearly explain the behavior of what's being seen and why the
> > solution works, per the review discussion.
> >
> > On Wed, Jul 27, 2011 at 3:46 PM, Juan L. Negron
> > <email address hidden>wrote:
> >
> > > Hi guys:
> > >
> > > This is also blocking the SI Team on the delivery of the
> > > cloudfoundry-server formula.
> > > We are experiencing the same issue.
> > > Any chance this can be prioritized high enough to get it fixed as soon
> as
> > > possible?
> > >
> > > Thanks,
> > >
> > > Juan
> > >
> > > --
> > > You received this bug notification because you are a member of Ensemble
> > > Composers, which is subscribed to the bug report.
> > > https://bugs.launchpad.net/bugs/810808
> > >
> > > Title:
> > > Formula install hook never completes
> > >
> > > Status in CloudFoundry:
> > > Confirmed
> > > Status in Ensemble:
> > > In Progress
> > >
> > > Bug description:
> > > Deploying lp:~gandelman-a/+junk/rabbitmq to a fresh system, ensemble
> > > fires the install hook but its exit seems to not be caught by ensemble
> > > *IF* it needs to install the package rabbitmq-server. ensemble status
> > > shows its state as 'null' On the machine, the hook is no longer
> > > running and log msgs generated by the hook show it is exiting
> > > successfully. ensemble.executor does not seem to catch the child
> > > process exiting and does not continue on to fire the 'start' hook:
> > >
> > > 2011-07-14 22:52:19,905: statemachine@DEBUG: unitworkflowstate:
> > execute
> > > action do_install
> > > 2011-07-14 22:52:19,905: hook.executor@DEBUG: Running hook:
> > > /var/lib/ensemble/units/rabbitmq-4/formula/hooks/install
> > > 2011-07-14 22:52:20,331: unit.hook.api@INFO: rabbitmq: Firing hook
> > > install.
> > > 2011-07-14 22:52:20,370: hook.output@INFO: Reading package lists...
> > > <......snip......>
> > > 2011-07-14 22:52:25,807: unit.hook.api@INFO: rabbitmq: Installation
> > > complete. 0
> > > 2011-07-14 22:52:26,237: unit.hook.api@INFO: rabbitmq: Hook install
> > > complete. Exiting 0
> > >
> > > However, if I destroy the service and deploy once again (without
> > > removing the packages), it successfully installs and moves onto
> > > starting:
> > >
> > > 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> > > transition install (None -> installed) {}
> > > 2011-07-14 23:57:47,461: statemachine@DEBUG: unitworkflowstate:
> > execute
> > > action do_install
> > > 2011-07-14 23:57:47,462: hook.executor@DEBUG: Running hook:
> > > /var/lib/ensemble/units/rabbitmq-16/formu...