VIP does not get set in hacluster

Bug #1813311 reported by Alexandros Soumplis
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Nova Cloud Controller Charm
Invalid
High
Unassigned

Bug Description

I have a nova-cloud-controller deployed with 3 units and set a VIP address. I deploy hacluster, make the relation and wait for cluster to complete and stabilize. If I have single-nova-consoleauth to false the hacluster does not configure the VIP resource:

root@juju-f9e12e-11-lxd-7:/var/log# crm configure show
node 1000: juju-f9e12e-10-lxd-7
node 1001: juju-f9e12e-11-lxd-7
node 1002: juju-f9e12e-9-lxd-7
primitive ping ocf:pacemaker:ping \
 params host_list=172.16.90.1 multiplier=100 \
 op monitor interval=5s
clone cl_ping ping \
 meta interleave=true
property cib-bootstrap-options: \
 have-watchdog=false \
 dc-version=1.1.18-2b07d5c5a9 \
 cluster-infrastructure=corosync \
 cluster-name=debian \
 no-quorum-policy=stop \
 stonith-enabled=false \
 last-lrm-refresh=1548408780
rsc_defaults rsc-options: \
 resource-stickiness=100

Setting single-nova-consoleauth to true, reconfigures the cluster and creates also the shared IP resource:

root@juju-f9e12e-11-lxd-7:/var/log/juju# crm configure show
node 1000: juju-f9e12e-10-lxd-7
node 1001: juju-f9e12e-9-lxd-7
node 1002: juju-f9e12e-11-lxd-7
primitive ping ocf:pacemaker:ping \
 params host_list=172.16.90.1 multiplier=100 \
 op monitor interval=5s
primitive res_nova_consoleauth ocf:openstack:nova-consoleauth \
 op monitor interval=5s
primitive res_nova_eth0_vip IPaddr2 \
 params ip=172.16.90.41 cidr_netmask=255.255.254.0 nic=eth0
primitive res_nova_haproxy lsb:haproxy \
 op monitor interval=5s
group grp_nova_vips res_nova_eth0_vip
clone cl_nova_haproxy res_nova_haproxy
clone cl_ping ping \
 meta interleave=true
colocation vip_consoleauth inf: res_nova_consoleauth grp_nova_vips
property cib-bootstrap-options: \
 have-watchdog=false \
 dc-version=1.1.18-2b07d5c5a9 \
 cluster-infrastructure=corosync \
 cluster-name=debian \
 no-quorum-policy=stop \
 stonith-enabled=false \
 last-lrm-refresh=1548425732
rsc_defaults rsc-options: \
 resource-stickiness=100

But then i get the error "Services not running that should be: nova-consoleauth". Setting it back to false leaves the cluster correctly configured. If it is of any help, the last change from true to false, produced the following error in the a unit's log file:
2019-01-25 14:38:13 DEBUG ha-relation-changed Resource 'vip_consoleauth' not found
2019-01-25 14:38:13 DEBUG ha-relation-changed Error performing operation: No such device or address

Revision history for this message
Pen Gale (pengale) wrote :

@soumplis: would you please post the bundle? We'll try to reproduce with the simplest possible setup of nova-cloud-controller and hacluster, but the original bundle would be useful, just in case that doesn't reproduce the issue.

Changed in charm-nova-cloud-controller:
status: New → Triaged
importance: Undecided → High
Changed in charm-nova-cloud-controller:
status: Triaged → Incomplete
Revision history for this message
Alexandros Soumplis (soumplis) wrote :

This is the current bundle of the installation with some sanitized data (IPs and domains)

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack nova-cloud-controller charm because there has been no activity for 60 days.]

Changed in charm-nova-cloud-controller:
status: Incomplete → Expired
Revision history for this message
Alexandros Soumplis (soumplis) wrote :

Is this resolved somehow ? Have you been able to reproduce ?

Changed in charm-nova-cloud-controller:
status: Expired → Triaged
Revision history for this message
James Page (james-page) wrote :

Setting back to new - this was pending information that has now been provided - bug still needs to be reproduced.

Changed in charm-nova-cloud-controller:
status: Triaged → New
Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

single-nova-consoleauth has been dropped from the charm in commit: b6e314077fa352ba58c346919a1e1cd4f6593226 (gerrit I2ac91b2bd92269b761befeb7563ad01cc5431151):

commit b6e314077fa352ba58c346919a1e1cd4f6593226
Author: James Page <email address hidden>
Date: Wed Mar 6 10:56:50 2019 +0000

    Drop support for single-nova-consoleauth

    Remove support for single-nova-consoleauth operation; this option
    managed a single instance of the nova-consoleauth process across
    a cluster nova-cloud-controller application using the hacluster
    charm. This proves somewhat racey on deployment as the ocf resource
    deep checks the operation of nova-consoleauth including connectivity
    to AMQP etc.. If the clustering of the service occurs before
    other principle relations have been completed, the resource will
    fail to start and the hook execution will spin, never returning.

    HA deployments should always use memcached to share tokens between
    instances of the nova-consolauth daemon; If the 'ha' relation is
    detected, then ensure that a memcache relation is then required
    for charm operation.

    To support evaluation of the memcache relation completeness
    the memcache specific code in InstanceConsoleContext was split out
    into a new memcache specific class RemoteMemcacheContext.

    Existing pacemaker resources will be deleted on upgrade; units will
    move into a blocked state until a relation is added to memcached.

    The nova-consoleauth service is resumed on upgrade to ensure that
    instances run on all nova-cloud-controller units.

    Change-Id: I2ac91b2bd92269b761befeb7563ad01cc5431151
    Closes-Bug: 1781620

Changed in charm-nova-cloud-controller:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.