fails to setup/start res_nova_consoleauth

[Nobuto Murata]

$ juju deploy -n2 cs:trusty/nova-cloud-controller
Added charm "cs:trusty/nova-cloud-controller-60" to the environment.

$ juju deploy cs:trusty/hacluster
Added charm "cs:trusty/hacluster-22" to the environment.

$ juju set nova-cloud-controller \
    console-access-protocol=novnc \
    single-nova-consoleauth=true \
    vip=10.5.0.200

$ juju ssh nova-cloud-controller/0 'sudo crm status'
Last updated: Fri Aug 14 09:38:20 2015
Last change: Fri Aug 14 09:35:17 2015 via crmd on juju-nobuto-machine-1
Stack: corosync
Current DC: juju-nobuto-machine-2 (168099853) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
4 Resources configured

Online: [ juju-nobuto-machine-1 juju-nobuto-machine-2 ]

 Resource Group: grp_nova_vips
     res_nova_eth0_vip (ocf::heartbeat:IPaddr2): Started juju-nobuto-machine-1
 Clone Set: cl_nova_haproxy [res_nova_haproxy]
     Started: [ juju-nobuto-machine-1 juju-nobuto-machine-2 ]

Failed actions:
    res_nova_consoleauth_start_0 (node=juju-nobuto-machine-1, call=71, rc=1, status=Timed Out, last-rc-change=Fri Aug 14 09:35:24 2015
, queued=20001ms, exec=0ms
): unknown error

[Nobuto Murata (nobuto)]

all-machines.log

[Ante Karamatić (ivoks)]

While this indeed looks like a bug, preferred way of ensuring token consistency is to relate nova-cloud-controller to memcached.

[Nobuto Murata (nobuto)]

@ante,

I assume you mean single-nova-consoleauth=false with memcached relation. single-nova-consoleauth=false has another issue as:
https://bugs.launchpad.net/charms/+source/nova-cloud-controller/+bug/1484605

[James Page (james-page)]

I think this is caused by some timing issues in the setup of the cluster; specifically:

    if True in [ra.startswith('ocf:openstack')
                for ra in resources.itervalues()]:
          apt_install('openstack-resource-agents')

does not happen until the hacluster charm parses relation data from the principle, at which point it might be installed and part of a corosync cluster with its peers; the cluster configuration is propagated from the lead unit, which already has things installed, and we end up with an ocf failure of this type.

I'd actually suggest we move any resource agent install to much earlier in the charm lifecycle, thus avoiding an inconsistent on-disk state with the actual cluster source configuration.

[James Page (james-page)]

Triaging as Low priority - this is not the recommended way of doing things with regards to HA nova-consoleauth - users should be using the memcache integration instead.

[Vern Hart (vern)]

After deployment at customer site we have to issue a crm resource cleanup on one of the nova-cloud-controller nodes to get consoleauth working. Not sure if it matters but we are using dns-ha=true in this deployment.

@james-page The standard FCE bundle templates do not related nova-cloud-controller with memcached and don't set single-nova-consoleauth (the default is true). Should we change this?