Comment 2 for bug 1838971

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/673456
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=8827e4f7f10d71027d9be4fbe650f9d1beccedeb
Submitter: Zuul
Branch: master

commit 8827e4f7f10d71027d9be4fbe650f9d1beccedeb
Author: Jiri Stransky <email address hidden>
Date: Tue Jul 30 10:57:54 2019 +0200

    Pacemaker resource upgrade tasks compatible with staged upgrade

    Add better idempotency checks on editing the pacemaker resources and
    fetching and re-tagging new images, which prevents the upgrade from
    failing. The latest status after staged upgrade looks like this:

    Online: [ controller-0 controller-1 controller-2 ]
    GuestOnline: [ galera-bundle-0@controller-0 galera-bundle-1@controller-1 galera-bundle-2@controller-2 rabbitmq-bundle-0@controller-0 rabbitmq-bundle-1@controller-1 redis-bundle-0@controller-0 redis-bundle-1@controller-1 ]

    Full list of resources:

     podman container set: galera-bundle [brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-mariadb:pcmklatest]
       galera-bundle-0 (ocf::heartbeat:galera): Master controller-0
       galera-bundle-1 (ocf::heartbeat:galera): Master controller-1
       galera-bundle-2 (ocf::heartbeat:galera): Master controller-2
     podman container set: rabbitmq-bundle [brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-rabbitmq:pcmklatest]
       rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started controller-0
       rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started controller-1
     podman container set: redis-bundle [brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-redis:pcmklatest]
       redis-bundle-0 (ocf::heartbeat:redis): Master controller-0
       redis-bundle-1 (ocf::heartbeat:redis): Slave controller-1
     ip-192.168.24.8 (ocf::heartbeat:IPaddr2): Started controller-0
     ip-10.0.0.106 (ocf::heartbeat:IPaddr2): Started controller-0
     ip-172.17.1.16 (ocf::heartbeat:IPaddr2): Started controller-0
     ip-172.17.1.23 (ocf::heartbeat:IPaddr2): Started controller-0
     ip-172.17.3.11 (ocf::heartbeat:IPaddr2): Started controller-0
     ip-172.17.4.25 (ocf::heartbeat:IPaddr2): Started controller-0
     podman container set: haproxy-bundle [brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-haproxy:pcmklatest]
       haproxy-bundle-podman-0 (ocf::heartbeat:podman): Started controller-0
       haproxy-bundle-podman-1 (ocf::heartbeat:podman): Started controller-1
       haproxy-bundle-podman-2 (ocf::heartbeat:podman): Stopped
     podman container: openstack-cinder-volume [brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-cinder-volume:pcmklatest]
       openstack-cinder-volume-podman-0 (ocf::heartbeat:podman): Started controller-1

    Failed Resource Actions:
    * rabbitmq_monitor_10000 on rabbitmq-bundle-0 'unknown error' (1): call=4861, status=Timed Out, exitreason='',
        last-rc-change='Mon Aug 5 10:37:51 2019', queued=0ms, exec=0ms
    * rabbitmq_monitor_10000 on rabbitmq-bundle-1 'unknown error' (1): call=42, status=Timed Out, exitreason='',
        last-rc-change='Mon Aug 5 10:15:55 2019', queued=0ms, exec=0ms

    This indicates that there are still issues we'll need to solve, but at
    least the upgrade passes now and we can keep solving the follow-up
    issues while the critical upgrade path is unblocked.

    Closes-Bug: #1838971
    Change-Id: I2e88dc34fa59624523de4c52a1873438c78e972f