ha restart bundles do not work on podman when the service runs on pacemaker remote

Bug #1838279 reported by Michele Baldessari
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Michele Baldessari

Bug Description

We blindly bind mount /etc/corosync/corosync.conf from the host to the container as that is required by pcs when running on pacemaker nodes.

When running on a pacemaker remote node, this is not needed and podman will actually barf on it because that file does not exist on a remote node:

        "stderr: error checking path \"/etc/corosync/corosync.conf\": stat /etc/corosync/corosync.conf: no such file or directory",

So we should not bind mount this when we run podman *and* we run on a pacemaker remote node.

Revision history for this message
Michele Baldessari (michele) wrote :
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)
Download full text (5.3 KiB)

Reviewed: https://review.opendev.org/673164
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=bdf742d05f90a57fdd08e73409d417bfcffdb8ab
Submitter: Zuul
Branch: master

commit bdf742d05f90a57fdd08e73409d417bfcffdb8ab
Author: Michele Baldessari <email address hidden>
Date: Mon Jul 29 17:02:03 2019 +0200

    Fix pcmk remote podman bundle restarts

    We blindly bind mount /etc/corosync/corosync.conf from the host to the
    container as that is required by pcs when running on pacemaker nodes
    when using pcs 0.9/pcmk 1.x.

    When running on a pacemaker remote node, this will actually barf on it
    because that file does not exist on a remote node:

            "stderr: error checking path \"/etc/corosync/corosync.conf\": stat /etc/corosync/corosync.conf: no such file or directory",

    This is not an issue with docker because docker simply creates non
    existing host bind mount points.
    So we should not bind mount this when we run podman because podman and
    HA only works on RHEL/CentOS 8 and it is not required there.

    Tested this on an Stein deployment with RHEL8 as OS and I correctly
    get a composable HA environment using pacemaker remote:
    overcloud-database-0 (ocf::pacemaker:remote): Started overcloud-controller-0
    overcloud-database-1 (ocf::pacemaker:remote): Started overcloud-controller-1
    overcloud-database-2 (ocf::pacemaker:remote): Started overcloud-controller-2
    overcloud-messaging-0 (ocf::pacemaker:remote): Started overcloud-controller-0
    overcloud-messaging-1 (ocf::pacemaker:remote): Started overcloud-controller-1
    overcloud-messaging-2 (ocf::pacemaker:remote): Started overcloud-controller-2
     podman container set: galera-bundle [192.168.24.1:8787/rhosp15/openstack-mariadb:pcmklatest]
       galera-bundle-0 (ocf::heartbeat:galera): Master overcloud-database-0
       galera-bundle-1 (ocf::heartbeat:galera): Master overcloud-database-1
       galera-bundle-2 (ocf::heartbeat:galera): Master overcloud-database-2
     podman container set: redis-bundle [192.168.24.1:8787/rhosp15/openstack-redis:pcmklatest]
       redis-bundle-0 (ocf::heartbeat:redis): Master overcloud-controller-0
       redis-bundle-1 (ocf::heartbeat:redis): Slave overcloud-controller-1
       redis-bundle-2 (ocf::heartbeat:redis): Slave overcloud-controller-2
     podman container set: rabbitmq-bundle [192.168.24.1:8787/rhosp15/openstack-rabbitmq:pcmklatest]
       rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-messaging-0
       rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-messaging-1
       rabbitmq-bundle-2 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-messaging-2
    ip-192.168.24.12 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
    ip-10.0.0.106 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
    ip-172.17.1.68 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
    ip-172.17.1.89 (ocf::heartbeat:IPaddr2): Started ove...

Read more...

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/675139

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/stein)
Download full text (5.4 KiB)

Reviewed: https://review.opendev.org/675139
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=e20d02a8b63e83deca47d3d53128c44b4e5ce629
Submitter: Zuul
Branch: stable/stein

commit e20d02a8b63e83deca47d3d53128c44b4e5ce629
Author: Michele Baldessari <email address hidden>
Date: Mon Jul 29 17:02:03 2019 +0200

    Fix pcmk remote podman bundle restarts

    We blindly bind mount /etc/corosync/corosync.conf from the host to the
    container as that is required by pcs when running on pacemaker nodes
    when using pcs 0.9/pcmk 1.x.

    When running on a pacemaker remote node, this will actually barf on it
    because that file does not exist on a remote node:

            "stderr: error checking path \"/etc/corosync/corosync.conf\": stat /etc/corosync/corosync.conf: no such file or directory",

    This is not an issue with docker because docker simply creates non
    existing host bind mount points.
    So we should not bind mount this when we run podman because podman and
    HA only works on RHEL/CentOS 8 and it is not required there.

    Tested this on an Stein deployment with RHEL8 as OS and I correctly
    get a composable HA environment using pacemaker remote:
    overcloud-database-0 (ocf::pacemaker:remote): Started overcloud-controller-0
    overcloud-database-1 (ocf::pacemaker:remote): Started overcloud-controller-1
    overcloud-database-2 (ocf::pacemaker:remote): Started overcloud-controller-2
    overcloud-messaging-0 (ocf::pacemaker:remote): Started overcloud-controller-0
    overcloud-messaging-1 (ocf::pacemaker:remote): Started overcloud-controller-1
    overcloud-messaging-2 (ocf::pacemaker:remote): Started overcloud-controller-2
     podman container set: galera-bundle [192.168.24.1:8787/rhosp15/openstack-mariadb:pcmklatest]
       galera-bundle-0 (ocf::heartbeat:galera): Master overcloud-database-0
       galera-bundle-1 (ocf::heartbeat:galera): Master overcloud-database-1
       galera-bundle-2 (ocf::heartbeat:galera): Master overcloud-database-2
     podman container set: redis-bundle [192.168.24.1:8787/rhosp15/openstack-redis:pcmklatest]
       redis-bundle-0 (ocf::heartbeat:redis): Master overcloud-controller-0
       redis-bundle-1 (ocf::heartbeat:redis): Slave overcloud-controller-1
       redis-bundle-2 (ocf::heartbeat:redis): Slave overcloud-controller-2
     podman container set: rabbitmq-bundle [192.168.24.1:8787/rhosp15/openstack-rabbitmq:pcmklatest]
       rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-messaging-0
       rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-messaging-1
       rabbitmq-bundle-2 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-messaging-2
    ip-192.168.24.12 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
    ip-10.0.0.106 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
    ip-172.17.1.68 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
    ip-172.17.1.89 (ocf::heartbeat:IPaddr2): Start...

Read more...

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 10.6.1

This issue was fixed in the openstack/tripleo-heat-templates 10.6.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.2.0

This issue was fixed in the openstack/tripleo-heat-templates 11.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.