pacemaker should have a binary version dependency on pacemaker libs

[Ante Karamatić]

Ubuntu 14.04

If system is running with pacemaker from the archive:

$ dpkg -l | grep 1.1.10+git20130802-1ubuntu2
ii libcib3 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - CIB
ii libcrmcluster4 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - CRM
ii libcrmcommon3 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - common CRM
ii libcrmservice1 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - crmservice
ii liblrmd1 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - lrmd
ii libpe-rules2 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - rules for P-Engine
ii libpe-status4 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - status for P-Engine
ii libpengine4 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - P-Engine
ii libstonithd2 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - stonith
ii libtransitioner2 1.1.10+git20130802-1ubuntu2 amd64 Pacemaker libraries - transitioner
ii pacemaker 1.1.10+git20130802-1ubuntu2 amd64 HA cluster resource manager
ii pacemaker-cli-utils 1.1.10+git20130802-1ubuntu2 amd64 Command line interface utilities for Pacemaker

$ sudo crm status
Last updated: Sat Oct 18 20:52:32 2014
Last change: Sat Oct 18 20:51:28 2014 via crmd on saturn
Stack: corosync
Current DC: saturn (2130706433) - partition with quorum
Version: 1.1.10-42f2063
1 Nodes configured
0 Resources configured

Online: [ saturn ]

And then one installs pacemaker (which pulls in pacemaker from -security):

$ sudo apt-get install pacemaker
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libccrtp0 libdbus-c++-1-0 libucommon6 libyate5.0.0 libzrtpcpp2
Use 'apt-get autoremove' to remove them.
The following packages will be upgraded:
  pacemaker
1 upgraded, 0 newly installed, 0 to remove and 59 not upgraded.
Need to get 364 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://hr.archive.ubuntu.com/ubuntu/ trusty-updates/main pacemaker amd64 1.1.10+git20130802-1ubuntu2.1 [364 kB]
Fetched 364 kB in 1s (197 kB/s)
(Reading database ... 638230 files and directories currently installed.)
Preparing to unpack .../pacemaker_1.1.10+git20130802-1ubuntu2.1_amd64.deb ...
Unpacking pacemaker (1.1.10+git20130802-1ubuntu2.1) over (1.1.10+git20130802-1ubuntu2) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up pacemaker (1.1.10+git20130802-1ubuntu2.1) ...
addgroup: The group `haclient' already exists as a system group. Exiting.
Warning: The home dir /var/lib/heartbeat you specified already exists.
The system user `hacluster' already exists. Exiting.

Restarting pacemaker results in havoc:

$ sudo /etc/init.d/pacemaker stop
Signaling Pacemaker Cluster Manager to terminate: [ OK ]
Waiting for cluster services to unload:^[[A.[ OK ]
$ sudo /etc/init.d/pacemaker start
Starting Pacemaker Cluster Manager: [ OK ]

$ sudo crm status
Last updated: Sat Oct 18 20:54:03 2014
Last change: Sat Oct 18 20:51:28 2014 via crmd on saturn
Stack: corosync
Current DC: NONE
1 Nodes configured
0 Resources configured

Node saturn (2130706433): UNCLEAN (offline)

From the syslog:

Oct 18 20:54:16 saturn crmd[23424]: warning: do_lrm_control: Failed to sign on to the LRM 2 (30 max) times
Oct 18 20:54:16 saturn crmd[23424]: warning: do_lrm_control: Failed to sign on to the LRM 3 (30 max) times
Oct 18 20:54:16 saturn crmd[23424]: warning: do_lrm_control: Failed to sign on to the LRM 4 (30 max) times

This is because pacemaker package depends on wrong library versions:

Package: pacemaker
Priority: optional
Section: admin
Installed-Size: 1564
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: Debian HA Maintainers <email address hidden>
Architecture: amd64
Version: 1.1.10+git20130802-1ubuntu2.1
Replaces: pacemaker-heartbeat, pacemaker-openais
Depends: libc6 (>= 2.14), libcfg6 (>= 2.3.0), libcib3 (>= 1.1.10+git20130802), libcmap4 (>= 2.3.0), libcorosync-common4 (>= 2.3.0), libcrmcluster4 (>= 1.1.10+git20130802), libcrmcommon3 (>= 1.1.10+git20130802), libcrmservice1 (>= 1.1.10+git20130802), libglib2.0-0 (>= 2.28.0), libgnutls26 (>= 2.12.17-0), liblrmd1 (>= 1.1.10+git20130802), libpam0g (>= 0.99.7.1), libpe-rules2 (>= 1.1.10+git20130802), libpe-status4 (>= 1.1.10+git20130802), libpengine4 (>= 1.1.10+git20130802), libplumb2 (>= 1.0.11+hg2754), libqb0, libstonithd2 (>= 1.1.10+git20130802), libtransitioner2 (>= 1.1.10+git20130802), python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2), libheartbeat2, resource-agents, adduser, corosync (>= 1.4.4-1) | heartbeat (>= 3.0), crmsh, pacemaker-cli-utils
Conflicts: pacemaker-heartbeat, pacemaker-openais

If now one updates libraries:

$ sudo apt-get install libcib3=1.1.10+git20130802-1ubuntu2.1 libcrmcluster4=1.1.10+git20130802-1ubuntu2.1 libcrmcommon3=1.1.10+git20130802-1ubuntu2.1 libcrmservice1=1.1.10+git20130802-1ubuntu2.1 liblrmd1=1.1.10+git20130802-1ubuntu2.1 libpe-rules2=1.1.10+git20130802-1ubuntu2.1 libpe-status4=1.1.10+git20130802-1ubuntu2.1 libpengine4=1.1.10+git20130802-1ubuntu2.1 libstonithd2=1.1.10+git20130802-1ubuntu2.1 libtransitioner2=1.1.10+git20130802-1ubuntu2.1 pacemaker-cli-utils=1.1.10+git20130802-1ubuntu2.1

And restarts pacemaker:

$ sudo /etc/init.d/pacemaker stop
Signaling Pacemaker Cluster Manager to terminate: [ OK ]
Waiting for cluster services to unload:[ OK ]
$ sudo /etc/init.d/pacemaker start
Starting Pacemaker Cluster Manager: [ OK ]

Cluster eventually resolves the problem:

$ sudo crm status
Last updated: Sat Oct 18 20:58:21 2014
Last change: Sat Oct 18 20:55:48 2014 via crmd on saturn
Stack: corosync
Current DC: saturn (2130706433) - partition with quorum
Version: 1.1.10-42f2063
1 Nodes configured
0 Resources configured

Online: [ saturn ]

And error messages from syslog are gone.

Pacemaker binary package should always depend on exact version of libraries its using.

[Robie Basak (racb)]

Thanks Ante. Though this is a regular update, not a security one.

[Ante Karamatić (ivoks)]

This is becoming serious issue cause it can easily be reproduced when upgrading charms.

[Rafael David Tinoco (rafaeldtinoco)]

Since there is a new SRU proposal on the following case:

https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1368737

I'll provide the dependencies fix together with that new SRU proposal.

I'll inform here when that bug (1368737) and this one (1382842) are addressed on that SRU proposal.

Thank you

Rafael Tinoco

[Rafael David Tinoco (rafaeldtinoco)]

It looks like the format chosen for SRU for this package :

pacemaker (1.1.10+git20130802-1ubuntu2.1) trusty
pacemaker (1.1.10+git20130802-1ubuntu2) trusty
pacemaker (1.1.10+git20130802-1ubuntu1) saucy

makes dh helpers not to calculate shlibs version properly:

$ fakeroot dh_makeshlibs -a -V

$ find . | grep shlib
./debian/libcrmcommon3/DEBIAN/shlibs
./debian/libpengine4/DEBIAN/shlibs
./debian/libpe-status4/DEBIAN/shlibs
./debian/libcrmservice1/DEBIAN/shlibs
./debian/libtransitioner2/DEBIAN/shlibs
./debian/libcib3/DEBIAN/shlibs
./debian/libpe-rules2/DEBIAN/shlibs
./debian/libcrmcluster4/DEBIAN/shlibs
./debian/libstonithd2/DEBIAN/shlibs
./debian/liblrmd1/DEBIAN/shlibs

$ cat ./debian/libcib3/DEBIAN/shlibs
libcib 3 libcib3 (>= 1.1.10+git20130802)

When the requested version, considering the following line in debian/control:

Package: pacemaker
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}, ${python:Depends}, ...

Should be:
libcib 3 libcib3 (>= 1.1.10+git20130802-1ubuntu2.2).

The fix for this problem will be released on next SRU, already proposed (for other case) in the following bug:

https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1368737

Please refer to that bug.

Thank you

Rafael Tinoco

[Rafael David Tinoco (rafaeldtinoco)]

Fixes (including this one) were attached to the following bug:

https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1368737

And are waiting for sponsorship.

[James Page (james-page)]

Theres probably a charm bug here as well as after discussion it turns out upgrading the hacluster charm also updates the pacemaker package with the security update; this is not intended behaviour.

[James Page (james-page)]

The right resolution to this problem is to pair the pacemaker binary package with exact versions of pacemaker libraries;

[Launchpad Janitor (janitor)]

This bug was fixed in the package pacemaker - 1.1.11-1ubuntu1

---------------
pacemaker (1.1.11-1ubuntu1) vivid; urgency=medium

  * Merge from Debian experimental, remaining changes:
    - d/control: Build-Depends on libcfg-dev.
    - Corosync's pacemaker plugin is disabled, hence not built:
      + d/libcrmcluster4-dev.install: Do not install plugin.h.
      + d/pacemaker.install,rules: Do not install pacemaker.lcrso.
    - Cherry picked fixes from upstream VCS:
      + Fix: services: Do not allow duplicate recurring op entries
      + High: lrmd: Merge duplicate recurring monitor operations
      + Fix: lrmd: Cancel recurring operations before stop action is executed
    - d/p/*: Refreshed.
  * Dropped changes, included in Debian:
    - d/control: Depend on libheartbeat2.
  * d/control: Ensure that pacemaker binary package uses matched binary
    versions of pacemaker libraries, avoiding upgrade problems (LP: #1382842).
  * d/p/Fix-lrmd-Prevent-glib-assert-triggered-by-timers-being.patch:
    Prevent glib assert triggered by timers being removed from mainloop more
    than once, cherry picked from upstream VCS (LP: #1368737).
  * d/control,compat,rules: Bump debhelper compat level to 9, drop use of
    hardening-wrapper.
  * d/control: Bumped Standards-Version, no changes.

pacemaker (1.1.11-1) experimental; urgency=medium

  * New upstream release
  * debian/patches/series: Disable 5 patches added for the last release
    as these are included in the upstream source now
  * debian/control: Bump Standards-Version
 -- James Page <email address hidden> Thu, 18 Dec 2014 10:26:17 +0000

[James Page (james-page)]

utopic and trusty fixes uploaded for SRU team review

[Brian Murray (brian-murray)]

Hello Ante, or anyone else affected,

Accepted pacemaker into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pacemaker/1.1.10+git20130802-1ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Ante, or anyone else affected,

Accepted pacemaker into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pacemaker/1.1.10+git20130802-4ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package pacemaker - 1.1.10+git20130802-4ubuntu3.1

---------------
pacemaker (1.1.10+git20130802-4ubuntu3.1) utopic; urgency=medium

  [ James Page ]
  * d/control: Ensure that pacemaker binary package uses matched binary
    versions of pacemaker libraries, avoiding upgrade problems (LP: #1382842).

  [ Rafael David Tinoco ]
  * d/p/Fix-lrmd-Prevent-glib-timers-removed-more-than-once.patch:
    Prevent glib assert triggered by timers being removed from mainloop more
    than once, cherry picked from upstream VCS (LP: #1368737).
 -- Rafael David Tinoco <email address hidden> Thu, 18 Dec 2014 10:14:38 +0000

[Chris J Arges (arges)]

The verification of the Stable Release Update for pacemaker has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package pacemaker - 1.1.10+git20130802-1ubuntu2.2

---------------
pacemaker (1.1.10+git20130802-1ubuntu2.2) trusty; urgency=medium

  [ James Page ]
  * d/control: Ensure that pacemaker binary package uses matched binary
    versions of pacemaker libraries, avoiding upgrade problems (LP: #1382842).

  [ Rafael David Tinoco ]
  * d/p/Fix-lrmd-Prevent-glib-timers-removed-more-than-once.patch:
    Prevent glib assert triggered by timers being removed from mainloop more
    than once, cherry picked from upstream VCS (LP: #1368737).
 -- Rafael David Tinoco <email address hidden> Thu, 18 Dec 2014 10:09:39 +0000