Apache RA doesn't create /var/run/apache2 [SRU]

[Brad Jones]

[Problem]
Apache OCF resource agent doesn't work when SSL module is enabled

[Impact]
This bug impacts all users that want to set up highly available Apache service with SSL, using Pacmaker or RedHat Cluster Suite tools.

[Development Fix]
Development branch isn't fixed yet either. Proposed solution is an upstream patch for this problem.

[Stable Fix]
Attached patch - resource-agents-apache.debdiff - is an oneliner, authored by upstream and part of the new upstream versions.

[Test Case]
To reproduce the problem, follow these steps on a clean 12.04 install:
1. apt-get install pacemaker apache2
2. service apache2 stop ; a2enmod ssl
3. update-rc.d -f apache2 remove
4. rm -rf /var/run/apache2 # or reboot
5. sed -i 's/START=no/START=yes/' /etc/default/corosync ; service corosync start
6. sleep 90 ; crm configure property stonith-enabled=false ; crm configure property no-quorum-policy=ignore
7. crm configure primitive apache2 ocf:heartbeat:apache params \
configfile="/etc/apache2/apache2.conf"
8. sleep 21 ; crm status | grep 'unknown exec error'
9. echo $?

Broken Behavior: last step (echo $?) will return 0, apache will not be running
Fixed Behavior: last step (echo $?) will return 1, apache will be running

[Regression Potential]
There are no known regressions. This patch impacts only this resource agent; and since it's not working now it would hardly introduce any worse side effects.

[Original Report]

Summary: the apache heartbeat RA that ships with both the universe package and PPA does not include a line to guarantee the existence of /var/run/apache2, which is usually done by apache2ctl. Since crm calls the RA, which in turn directly calls the apache2 daemon, this bit of housekeeping never gets done.

I originally raised this in https://github.com/ClusterLabs/resource-agents/issues/70 but then realized this PPA isn't in sync with what they're doing there. (I'm guessing this package is downstream from that?)

The line in question, at that remote repo, is https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/apache#L177

I discovered this bug in the course of setting up a Linux HA cluster locally. The line I'm proposing be added is also absent from the universe cluster-agents package that ships with lucid. (See http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/lucid/cluster-agents/lucid-proposed/view/head:/heartbeat/apache#L399)

I'd submit a patch but I am not much up to speed on bzr... it's one line though so not too complex.

[Adam Gandelman (gandelman-a)]

Brad-

Thanks for reporting this. I'm curious as to what PPA you are refering to? Does the issue exist in the cluster-agents package currently available in Precise (1:1.0.4-0ubuntu2) ? It's a bit late to fix this for release, but if you can confirm it affects the current version to be shipped in Precise we can queue up an SRU to fix it shortly after Precise is released.

[Brad Jones (brad-jones)]

The PPA is at: https://launchpad.net/~ubuntu-ha-maintainers/+archive/ppa

This bug persists in the Precise package at: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/precise/cluster-agents/precise/view/head:/heartbeat/apache#L405

[Ante Karamatić (ivoks)]

This debdiff patch provides upstream solution for this problem. Binary was built using pbuilder and it's been tested. It works as designed.

[Ante Karamatić (ivoks)]

Fix committed to ubuntu-ha-maintainers ppa for lucid.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "resource-agents-apache.debdiff" of this bug report has been identified as being a patch in the form of a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

[Bryce Harrington (bryce)]

Heya, first off, thanks for this contribution to the LTS, it's much
appreciated!

As it happens, Ubuntu Precise has now progressed into Final Freeze.
Right now changes to the distro are limited to just ultra-critical
release bugs that the release team has flagged as priorities. Most
other bugs, including this one, need to now be handled as Stable
Release Updates (SRUs) or left until the next release (Q-series) opens
for development.

For reference, here is the SRU Policy Doc:
  https://wiki.ubuntu.com/StableReleaseUpdates

I've looked at your patch and think it may qualify as an SRU, but
there's a bit extra paperwork, which helps establish the justification
for the change and makes the change easier for reviewers. Would you
mind helping by filling in some blanks?

I've pasted in the standard SRU template into the bug description.
Click the yellow pencil icon to the right of 'Bug Description', and fill
in the empty sections.

Also, while the patch itself looks ok, there are a few changes necessary
to target it as an SRU:

 1. In the changelog entry, target the upload to 'precise-proposed'
    rather than just 'precise'.

 2. The version number should increment by a point rather than a full
    number. Examples:

     Currently in precise Proposed for precise-proposed
     foobar-1.2-1 ---> foobar-1.2-1ubuntu0.1
     foobar-1.2-1ubuntu2 ---> foobar-1.2-1ubuntu2.1
     foobar-1.2~dfsg.1-1 ---> foobar-1.2~dfsg.1-1ubuntu0.1
     foobar-1.2-1ubuntu1.5 ---> foobar-1.2-1ubuntu1.6

 3. Make certain the correct bug # is listed in the changelog entry
    description, in this format: (LP: #123456)

 4. Doublecheck the maintainer is set to Ubuntu. Easy to do, just run
    `update-maintainer` and it'll fix it.

[Bryce Harrington (bryce)]

Please re-subscribe the ubuntu-sponsors team to the bug report once the above has been done.

[Ante Karamatić (ivoks)]

Regarding comment #6:

1) fixed
2) fixed
3) it was already ok
4) it was already ok

Bug description is updated with instructions how to reproduce the problem, with both expected and broken results.

[Martin Pitt (pitti)]

Hello Brad, or anyone else affected,

Accepted resource-agents into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Stéphane Graber (stgraber)]

Ante: still not working... ran two VMs with the exact same setup, one with -proposed, one without, both return 1 and no apache2 running.

I don't really doubt the SRU as the patch is fairly obvious, but I won't mark it verification-done until I have a working testcase.

[Ante Karamatić (ivoks)]

Actually, if it returns '1', apache is running. This was due the fact that /var/run/apache2 existed after install. So, after disabling apache2 start on boot, one should reboot or delete /var/run/apache2 to simulate clean start.

[Launchpad Janitor (janitor)]

This bug was fixed in the package resource-agents - 1:3.9.2-5ubuntu6

---------------
resource-agents (1:3.9.2-5ubuntu6) quantal; urgency=low

  [ Ante Karamatic ]
  * debian/patches/apache-var-run.patch:
    - upstream commit 0c128518d2
    - apache OCF doesn't create /var/run/apache2 before starting
    - (LP: #980180)
 -- Andres Rodriguez <email address hidden> Mon, 11 Jun 2012 10:27:25 -0400

[Stéphane Graber (stgraber)]

Marking verification-done based on these few tests (no obvious regression) and the obvious fix. I managed to get it running after an hour of messing with it, so something in the testcase must still be somehow missing, though in the end it did work, so I think that's good enough.

[Launchpad Janitor (janitor)]

This bug was fixed in the package resource-agents - 1:3.9.2-5ubuntu4.1

---------------
resource-agents (1:3.9.2-5ubuntu4.1) precise-proposed; urgency=low

  * debian/patches/apache-var-run.patch:
    - upstream commit 0c128518d2
    - apache OCF doesn't create /var/run/apache2 before starting
    - (LP: #980180)
 -- Ante Karamatic <email address hidden> Tue, 17 Apr 2012 09:45:12 +0200

[Rolf Leggewie (r0lf)]

quantal has seen the end of its life and is no longer receiving any updates. Marking the quantal task for this ticket as "Won't Fix".

[Steve Langasek (vorlon)]

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release