This is newton openstack AFAIKs.

var/log/juju/unit-hacluster-nova-cloud-controller-0.log:2017-11-16 12:18:10 DEBUG install Get:49 http://archive.ubuntu.com/ubuntu xenial/universe ...


Which charm release though: 17.02, 17.08 or something else? Could you please paste the relevant "juju status" with charm revision numbers? Otherwise it's hard to say where this may be coming from.

On a "bad" hacluster there's a CIB rendered that doesn't pass DTD validation.

Updated CIB does not validate against pacemaker-2.4 schema/dtd

See below:


hacluster unit log (bad):
Nov 16 12:22:53 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_process_request: Completed cib_apply_diff operation for section 'all': Update does not conform to the configured schema (rc=-203, origin=juju-37c0a4-2-lxd-4/cibadmin/2, version=0.8.16)
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      Diff: --- 0.8.16 2
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      Diff: +++ 0.9.0 (null)
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      +  /cib:  @epoch=9, @num_updates=0
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      ++ /cib/configuration/resources:  <group id="grp_nova_vips"/>
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Expecting an element meta_attributes, got nothing
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Invalid sequence in interleave
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Element group failed to validate content
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Element resources has extra content: group
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_perform_op:      Updated CIB does not validate against pacemaker-2.4 schema/dtd
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_diff_notify:     Update (client: cibadmin, call:2): 0.8.16 -> 0.9.0 (Update does not conform to the configured schema)
Nov 16 12:23:09 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_process_request: Completed cib_apply_diff operation for section 'all': Update does not conform to the configured schema (rc=-203, origin=juju-37c0a4-0-lxd-4/cibadmin/2, version=0.8.16)
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      Diff: --- 0.8.16 2
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      Diff: +++ 0.9.0 (null)
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      +  /cib:  @epoch=9, @num_updates=0
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:     info: cib_perform_op:      ++ /cib/configuration/resources:  <clone id="cl_nova_haproxy"/>
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Expecting an element meta_attributes, got nothing
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Invalid sequence in interleave
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Element clone failed to validate content
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:    error: xml_log:     Element resources has extra content: clone
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_perform_op:      Updated CIB does not validate against pacemaker-2.4 schema/dtd
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_diff_notify:     Update (client: cibadmin, call:2): 0.8.16 -> 0.9.0 (Update does not conform to the configured schema)
Nov 16 12:23:10 [169497] juju-37c0a4-1-lxd-4        cib:  warning: cib_process_request: Completed cib_apply_diff operation for section 'all': Update does not conform to the configured schema (rc=-203, origin=juju-37c0a4-0-lxd-4/cibadmin/2, version=0.8.16)
Nov 16 12:36:09 [169502] juju-37c0a4-1-lxd-4       crmd:   notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]


nova-cloud-controller unit (bad):

017-11-16 12:22:38 DEBUG juju-log ha:86: Configuring Groups: {'grp_nova_vips': 'res_nova_eth0_vip res_nova_eth1_vip'}
2017-11-16 12:22:38 DEBUG ha-relation-changed ERROR: grp_nova_vips refers to missing object res_nova_eth0_vip
2017-11-16 12:22:38 DEBUG ha-relation-changed ERROR: grp_nova_vips refers to missing object res_nova_eth1_vip
2017-11-16 12:22:38 DEBUG ha-relation-changed ERROR: syntax in group: Expected at least 2 arguments near <group> parsing 'group grp_nova_vips'
2017-11-16 12:22:38 DEBUG ha-relation-changed Call cib_apply_diff failed (-203): Update does not conform to the configured schema
2017-11-16 12:22:38 DEBUG ha-relation-changed <cib crm_feature_set="3.0.10" validate-with="pacemaker-2.4" epoch="9" num_updates="0" admin_epoch="0" cib-last-written="Thu Nov 16 12:22:38 2017" update-origin="juju-37c0a4-1-lxd-4" update-client="cibadmin" update-user="root" have-quorum="1" dc-uuid="1000">