No "upstart" support in cluster-glue-1.0.5-1

[Henk Bokhoven]

Binary package hint: cluster-glue

In package cluster-glue-1.0.5-1 for Ubuntu Lucid (10.04.2) there is no support for the in that release introduced "upstart".
Still only "lsb". This results in messages like this in /var/log/corosync/corosync.log:

"Rather than invoking init scripts through /etc/init.d, use the service(8)"

"Upstart" with Pacemaker seems to be possible, refering to this blogpost by Florian Haas of Linbit / DRBD:
http://fghaas.wordpress.com/2010/09/15/upstart-support-coming-to-pacemaker/

I would like to see a Lucid package-update that FULLY supports this new functionality introduced in 10.04. TIA!

[Ante Karamatić (ivoks)]

U Uto, 22. 03. 2011., u 09:59 +0000, Henk Bokhoven je napisao/la:

> "Upstart" with Pacemaker seems to be possible, refering to this blogpost by Florian Haas of Linbit / DRBD:
> http://fghaas.wordpress.com/2010/09/15/upstart-support-coming-to-pacemaker/

Have you tried defining a resource as upstart:apache2? For example

primitive upstart:apache2

[Henk Bokhoven (bokh)]

Yes I did.
Replacing "primitive mysqld lsb:mysql" with "primitive mysqld upstart:mysql" gives an error:

ERROR: upstart:mysql: could not parse meta-data:
ERROR: upstart:mysql: no such resource agent

[Ante Karamatić (ivoks)]

U Uto, 22. 03. 2011., u 13:04 +0000, Henk Bokhoven je napisao/la:

> Yes I did.
> Replacing "primitive mysqld lsb:mysql" with "primitive mysqld upstart:mysql" gives an error:
>
> ERROR: upstart:mysql: could not parse meta-data:
> ERROR: upstart:mysql: no such resource agent

Ah, my mistake. This feature was developed post-lucid. There is a PPA
which you can use:

https://launchpad.net/~ubuntu-ha-maintainers/+archive/ppa

[Henk Bokhoven (bokh)]

Thanks Ante, but so far this does not seem to work. Same error as in post #2.

I added the PPA to /etc/apt/sources.list and ran the update on a test-cluster with 2 nodes.
When using "upstart:mysql" instead of "lsb:mysql" the whole cluster is offline, so I had to get back to "lsb" (again).

List of the old, default Lucid cluster-related packages:

# grep -i cluster packages1
ii cluster-agents 1:1.0.3-2ubuntu1 The reusable cluster components for Linux HA
ii cluster-glue 1.0.5-1 The reusable cluster components for Linux HA
ii corosync 1.2.0-0ubuntu1 Standards-based cluster framework (daemon an
ii libcluster-glue 1.0.5-1 The reusable cluster components for Linux HA
ii libcorosync4 1.2.0-0ubuntu1 Standards-based cluster framework (libraries
ii pacemaker 1.0.8+hg15494-2ubuntu2 HA cluster resource manager

List of now new(er) packages from the above mentioned PPA:

# grep -i cluster packages2
ii cluster-agents 1:1.0.3-3~ppa1 The reusable cluster components for Linux HA
ii cluster-glue 1.0.6-1ubuntu1ppa1 The reusable cluster components for Linux HA
ii corosync 1.2.1-1ubuntu1~ppa1 Standards-based cluster framework (daemon an
ii libcluster-glue 1.0.6-1ubuntu1ppa1 The reusable cluster components for Linux HA
ii libcorosync4 1.2.1-1ubuntu1~ppa1 Standards-based cluster framework (libraries
ii pacemaker 1.0.9.1-2ubuntu2~ppa1 HA cluster resource manager

So "cluster-glue" is updated from 1.0.5-1 to 1.0.6-1, but "upstart" is still not functioning.

[Ante Karamatić (ivoks)]

U Uto, 22. 03. 2011., u 14:09 +0000, Henk Bokhoven je napisao/la:

> I added the PPA to /etc/apt/sources.list and ran the update on a test-cluster with 2 nodes.
> When using "upstart:mysql" instead of "lsb:mysql" the whole cluster is offline, so I had to get back to "lsb" (again).

lsb won't work. upstart will and it does. It's been tested and I'm using
it on dozen locations. Paste the output of 'crm configure show' and 'crm
status'.

[Henk Bokhoven (bokh)]

I'm still under the impression "lsb" is the only valid primitive in the current set of packages I've got installed.
Here's the output you requested, along with a "crm configure edit" when I try to replace "lsb" with "upstart".

# crm configure show
node db1 \
        attributes standby="off"
node db3 \
        attributes standby="off"
primitive drbd_mysql ocf:linbit:drbd \
        params drbd_resource="mysql" \
        op monitor interval="15s"
primitive fs_mysql ocf:heartbeat:Filesystem \
        params device="/dev/drbd/by-res/mysql" directory="/mnt/" fstype="ext4"
primitive ip_mysql ocf:heartbeat:IPaddr2 \
        params ip="192.168.0.166" nic="eth0"
primitive mysqld lsb:mysql \
        op start interval="0" timeout="120s" \
        op stop interval="0" timeout="120s" \
        op monitor interval="30s" \
        meta target-role="Started"
group mysql_group fs_mysql ip_mysql mysqld \
        meta target-role="Started"
ms ms_drbd_mysql drbd_mysql \
        meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
location cli-prefer-mysqld mysqld \
        rule $id="cli-prefer-rule-mysqld" inf: #uname eq db1
colocation mysql_on_drbd inf: mysql_group ms_drbd_mysql:Master
order mysql_after_drbd inf: ms_drbd_mysql:promote mysql_group:start
property $id="cib-bootstrap-options" \
        dc-version="1.0.9-unknown" \
        cluster-infrastructure="openais" \
        expected-quorum-votes="2" \
        no-quorum-policy="ignore" \
        stonith-enabled="false" \
        last-lrm-refresh="1299838408"
rsc_defaults $id="rsc-options" \
        resource-stickiness="100"

=================================================================

# crm status
============
Last updated: Tue Mar 22 16:09:44 2011
Stack: openais
Current DC: db3 - partition with quorum
Version: 1.0.9-unknown
2 Nodes configured, 2 expected votes
2 Resources configured.
============

Online: [ db1 db3 ]

 Master/Slave Set: ms_drbd_mysql
     Masters: [ db1 ]
     Slaves: [ db3 ]
 Resource Group: mysql_group
     fs_mysql (ocf::heartbeat:Filesystem): Started db1
     ip_mysql (ocf::heartbeat:IPaddr2): Started db1
     mysqld (lsb:mysql): Started db1

=================================================================

As said, when I replace "lsb" with "upstart" and try to save this new config I get these errors:

# crm configure edit
element primitive: Relax-NG validity error : Invalid sequence in interleave
element primitive: Relax-NG validity error : Element primitive failed to validate content
Relax-NG validity error : Extra element primitive in interleave
element primitive: Relax-NG validity error : Element group failed to validate content
element master: Relax-NG validity error : Element resources has extra content: master
element cib: Relax-NG validity error : Element cib failed to validate content
crm_verify[2267]: 2011/03/22_16:11:04 ERROR: main: CIB did not pass DTD/schema validation
Errors found during check: config not valid
ERROR: update:mysql: could not parse meta-data:
ERROR: update:mysql: no such resource agent
Do you still want to commit? n

[Ante Karamatić (ivoks)]

U Uto, 22. 03. 2011., u 15:15 +0000, Henk Bokhoven je napisao/la:

> ERROR: update:mysql: could not parse meta-data:
> ERROR: update:mysql: no such resource agent

upstart, not update.

[Henk Bokhoven (bokh)]

Sorry for the typo, my bad... However the issue remains:

# crm configure edit
ERROR: upstart:mysql: could not parse meta-data:
ERROR: upstart:mysql: no such resource agent
Do you still want to commit? n

What is causing this? IMHO cluster-glue, hence the bug-report.
But maybe now is a good time to refrase my request:
Since the default Lucid-packages obviously do not support "upstart", but only "lsb", WHAT exact packages from what PPA do I need to run a Pacemaker/Corsync/OpenAIS cluster with full "upstart" support on Ubuntu Lucid 10.04.2 LTS?

[Henk Bokhoven (bokh)]

FYI - this almost seems like a duplicate of bug #659510 - check comment #12:
https://bugs.launchpad.net/ubuntu/+source/cluster-glue/+bug/659510/comments/12

So up until now the _upstart resource agents_ are missing on my system, hence the "no such resource agent".

[Andres Rodriguez (andreserl)]

Henk,

cluster-glue 1.0.5 does not have support for upstart because the upstart support was added post that release of cluster-glue. This however, is fixed in Maverick by manually patching cluster-glue.

However, upstream officially released the upstart lrmd plugin in cluster-glue 1.0.7, with the following announcement:

- (note for Ubuntu users) the new ubuntu startup lrmd
                 plugin is included but unfortunately not functional,
                 it will have to wait for one of the next releases

However, in Ubuntu, cluster-glue has been patched to make it functional. Unfortunately, upstream has not accepted the patch that fixes this.

Hope this answers your questions.

[Ante Karamatić (ivoks)]

U Uto, 22. 03. 2011., u 16:53 +0000, Henk Bokhoven je napisao/la:

> FYI - this almost seems like a duplicate of bug #659510 - check comment #12:
> https://bugs.launchpad.net/ubuntu/+source/cluster-glue/+bug/659510/comments/12
>
> So up until now the _upstart resource agents_ are missing on my system,
> hence the "no such resource agent".

You are right. Our source package in the PPA was compiling everything
needed, but it didn't install upstart plugin cause of the bug in the
package build. If you update your packages from the PPA, you'll get
everything that you need ;)

P.S. packages are built, but it may take few more minutes before they
are published

[Henk Bokhoven (bokh)]

Thanks! "upstart" is now accepted instead of "lsb" (after a reboot).
upstart resource agent(s) are in place at /usr/lib/heartbeat/plugins/RAExec/upstart.so

The corosync log-messages are now showing:

Mar 22 19:37:13 db1 lrmd: [969]: info: RA output: (mysqld:start:stdout) Since the script you are attempting to invoke has been converted to an
Mar 22 19:37:13 db1 lrmd: [969]: info: RA output: (mysqld:start:stdout) Upstart job, you may also use the start(8) utility, e.g. start mysql

From my point of view this bug can get status "resolved" / "closed".
Once again thanks for your efforts and updates!