ensemble should make it easy to share data between hooks

[Juan L. Negron]

In Ensemble, once a relationship is made, the hooks are called that allow the creation of relationship bits via relation-set <blah>=<value of blah>.

During the install script, there is no apparent mechanism by which to start saving variables and their corresponding values. Maybe this is a design issue but, there are formulas the need to have information preserved at the install level and not at the relationship level.

A good example of this is tomcat: In tomcat, the server.xml file needs to be created with static members that require a unique identifier as well as a unique id for each node of the "cluster".

The problem with this is that when the peer-relation hook is called it will only have the necessary information of the remote host but not that of itself.

I can ( on install ) write all of the necessary values to a file and work from there but, I believe an equivalent command to relation-set/get/list that can be used during the install hook-script would be very useful. Of course the information needs to also be available during the relation-get/set/list commands.

This information ( or fact-db ) will become increasingly necessary as we continue to develop and deploy formulas that require uuid's and clusternames for their proper clustering ( ie: cassandra, cloudfoundry, tomcat, mondgodb, etc.)

-Juan

[Gustavo Niemeyer (niemeyer)]

I don't think I understand the problem. Hooks reflect specific events, and do not enforce in any way what you're supposed to be doing within them. The install hook, by design, is called just once when the formula is first installed, and in general is a convenient location to run installation of software. That said, there's nothing preventing you from running logic in any of the other hooks. If you can only install/start the software when a relation is established, you should feel free to do so.

[Juan L. Negron (negronjl)]

I am working around the issue of shared data available from the install hook
by using facter and my own facter-plugins. I still think it is a good idea
for ensemble to provide a mechanism by which the information can be
stored/shared/accessed from the install hook and not necessarily wait for
the relationships to start.

-Juan

On Tue, Jul 5, 2011 at 7:14 PM, Gustavo Niemeyer <email address hidden>wrote:

> I don't think I understand the problem. Hooks reflect specific events,
> and do not enforce in any way what you're supposed to be doing within
> them. The install hook, by design, is called just once when the formula
> is first installed, and in general is a convenient location to run
> installation of software. That said, there's nothing preventing you
> from running logic in any of the other hooks. If you can only
> install/start the software when a relation is established, you should
> feel free to do so.
>
>
> ** Changed in: ensemble
> Status: New => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/806098
>
> Title:
> fact repository for Ensemble
>
> Status in Ensemble:
> Incomplete
>
> Bug description:
> In Ensemble, once a relationship is made, the hooks are called that
> allow the creation of relationship bits via relation-set <blah>=<value
> of blah>.
>
> During the install script, there is no apparent mechanism by which to
> start saving variables and their corresponding values. Maybe this is
> a design issue but, there are formulas the need to have information
> preserved at the install level and not at the relationship level.
>
> A good example of this is tomcat: In tomcat, the server.xml file
> needs to be created with static members that require a unique
> identifier as well as a unique id for each node of the "cluster".
>
> The problem with this is that when the peer-relation hook is called it
> will only have the necessary information of the remote host but not
> that of itself.
>
> I can ( on install ) write all of the necessary values to a file and
> work from there but, I believe an equivalent command to relation-
> set/get/list that can be used during the install hook-script would be
> very useful. Of course the information needs to also be available
> during the relation-get/set/list commands.
>
> This information ( or fact-db ) will become increasingly necessary as
> we continue to develop and deploy formulas that require uuid's and
> clusternames for their proper clustering ( ie: cassandra,
> cloudfoundry, tomcat, mondgodb, etc.)
>
> -Juan
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ensemble/+bug/806098/+subscriptions
>

[Gustavo Niemeyer (niemeyer)]

It's still not entirely clear what you're looking for. You're using facters for what, more precisely, and how's that connected with relationships?

relation-set is specific to the relation being changed, so it's impossible to have it during the install hook,
because there are no relations established at this point.

[Kapil Thangavelu (hazmat)]

Excerpts from Gustavo Niemeyer's message of Tue Jul 05 19:34:48 UTC 2011:
> It's still not entirely clear what you're looking for. You're using
> facters for what, more precisely, and how's that connected with
> relationships?
>
> relation-set is specific to the relation being changed, so it's impossible to have it during the install hook,
> because there are no relations established at this point.
>

It sounds like this bug is really about unit private data. Effectively a place to stash runtime information in zk across hooks. What we've discussed before in this context is hooks are free to store data locally on disk in whatever format is convienent. Our zookeeper storage is really specific to coordination cases, which span cross machine communication. All that said, i think it might be fine to establish some sort of k/v store and cli api for the unit against say a sqlite db.

[Juan L. Negron (negronjl)]

relationships are not related to facts. I just see the need for facts so I
can save/precompute information at the install hook that can then be shared
with the other hooks ( or be available at the
relation-name-relationship-create/change level ).

ie: tomcat static members for it's cluster configuration require a
clustername that should be shared at install ( when the configuration takes
place ). Having this information handy could be useful when add-node is
used and the peer relationship hook gets called. The information created at
the install hook would now be available at the relationship-create hook.

Once I have the formula finished, I'll push it to the branch and you can
have a look at what I'm doing with it. Maybe that will clarify some things.

-Juan

On Tue, Jul 5, 2011 at 8:34 PM, Gustavo Niemeyer <email address hidden>wrote:

> It's still not entirely clear what you're looking for. You're using
> facters for what, more precisely, and how's that connected with
> relationships?
>
> relation-set is specific to the relation being changed, so it's impossible
> to have it during the install hook,
> because there are no relations established at this point.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/806098
>
> Title:
> fact repository for Ensemble
>
> Status in Ensemble:
> Incomplete
>
> Bug description:
> In Ensemble, once a relationship is made, the hooks are called that
> allow the creation of relationship bits via relation-set <blah>=<value
> of blah>.
>
> During the install script, there is no apparent mechanism by which to
> start saving variables and their corresponding values. Maybe this is
> a design issue but, there are formulas the need to have information
> preserved at the install level and not at the relationship level.
>
> A good example of this is tomcat: In tomcat, the server.xml file
> needs to be created with static members that require a unique
> identifier as well as a unique id for each node of the "cluster".
>
> The problem with this is that when the peer-relation hook is called it
> will only have the necessary information of the remote host but not
> that of itself.
>
> I can ( on install ) write all of the necessary values to a file and
> work from there but, I believe an equivalent command to relation-
> set/get/list that can be used during the install hook-script would be
> very useful. Of course the information needs to also be available
> during the relation-get/set/list commands.
>
> This information ( or fact-db ) will become increasingly necessary as
> we continue to develop and deploy formulas that require uuid's and
> clusternames for their proper clustering ( ie: cassandra,
> cloudfoundry, tomcat, mondgodb, etc.)
>
> -Juan
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ensemble/+bug/806098/+subscriptions
>

[Clint Byrum (clint-fewbar)]

Juan, facter is pretty much made for this very purpose. I actually like the way you've made use of it.

This data never needs to leave the local machine. Its just a local scratch pad, so putting it in ZK is a little bit weird.

What one needs to do is have a set of variables that are always accessible, which are controllable by each hook. I think this is a pretty common need, and will be used a lot with things like config settings.. where you may just store the settings in facts, and then re-run a common template regeneration mechanism. That sounds an awful lot like the way puppet does config management... and it also uses facter.

We should be able to develop a much lighter weight command that just stores keys/values locally and retrieves them when needed. I'd put this pretty low on the priority list though, as its not that terrible to depend on facter inside formulas.

[Stuart Bishop (stub)]

I need to keep track of unique ids allocated to units in a peer relationship.

Per Juan's and my own use cases, local storage is not suitable as the unit storing the information can disappear. Without access to a central store, failover can only happen if all the required state can be regenerated from scratch.