OCF: neutron-agent-l3 meta-data broken

[Bogdan Dobrelya]

# usr/lib/ocf/resource.d/mirantis/neutron-agent-l3 meta-data
is broken now and return errors:
#OCF_ROOT=/usr/lib/ocf /usr/lib/ocf/resource.d/mirantis/neutron-agent-l3 meta-data

/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 371: get_pid_list_for_ns_list: command not found
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 383: get_pid_list_for_ns_list: command not found
2014-10-01 08:15:31,695 - INFO - Started: /usr/bin/q-agent-cleanup.py --agent=l3 --cleanup-ports
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 392: syntax error near unexpected token `}'
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 392: `}'

Puppet writes to its stderr
ERROR: ocf:mirantis:neutron-agent-l3: meta-data is no good XML

And in puppet logs there are fatal error:
2014-09-30 15:29:59 ERR (/Stage[main]/Neutron::Agents::L3/Service[neutron-l3]) /etc/puppet/modules/corosync/lib/puppet/provider/service/pacemaker.rb:70:in `get_service_hash'
2014-09-30 15:29:59 ERR (/Stage[main]/Neutron::Agents::L3/Service[neutron-l3]) Could not evaluate: undefined method `attributes' for nil:NilClass

reproduced at {"build_id": "2014-09-29_19-01-25", "ostf_sha": "727cd3cc93e255635fef13c80baee649ced075be", "build_number": "536", "auth_required": true, "api": "1.0", "nailgun_sha": "618d45b0cceef67b4da616167cab3161d55e9b52", "production": "docker", "fuelmain_sha": "2aa00ddbcfc9b32e01bdca8f63f759129c9c2b54", "astute_sha": "a3e5da62af91b99f958ab958161d3dcec09c657b", "feature_groups": ["mirantis"], "release": "6.0", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-09-29_19-01-25", "ostf_sha": "727cd3cc93e255635fef13c80baee649ced075be", "build_number": "536", "api": "1.0", "nailgun_sha": "618d45b0cceef67b4da616167cab3161d55e9b52", "production": "docker", "fuelmain_sha": "2aa00ddbcfc9b32e01bdca8f63f759129c9c2b54", "astute_sha": "a3e5da62af91b99f958ab958161d3dcec09c657b", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "73f2e3ca9dca9809de31471e15b58c634a81254b"}}}, "fuellib_sha": "73f2e3ca9dca9809de31471e15b58c634a81254b"}

Neutron HA gre, Ubuntu.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/125085

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/125086

[Bogdan Dobrelya (bogdando)]

Suggested fixes are not enough

[Bogdan Dobrelya (bogdando)]

The error in bug description is not related to CIB queries, the root cause is:
#/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3 meta-data

/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 29: /lib/heartbeat/ocf-shellfuncs: No such file or directory
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 371: get_pid_list_for_ns_list: command not found
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 383: get_pid_list_for_ns_list: command not found
2014-10-01 08:06:23,907 - INFO - Started: /usr/bin/q-agent-cleanup.py --agent=l3 --cleanup-ports
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 392: syntax error near unexpected token `}'
/usr/lib/ocf/resource.d/mirantis/neutron-agent-l3: line 392: `}'

(Note, that "2014-10-01 08:06:23,907 - INFO - Started: /usr/bin/q-agent-cleanup.py --agent=l3 --cleanup-ports" already fixed in master branch by https://review.openstack.org/#/c/125098/)

[Bogdan Dobrelya (bogdando)]

RC is a typo
cbe6de69 deployment/puppet/neutron/files/ocf/neutron-agent-l3 (Sergey Vasilenko 2014-09-22 15:36:42 +0400 65) : ${OCF_RESKEY_rescheduling_tries=${OCF_RESKEY_rescheduling_tries_default}5

should be ${OCF_RESKEY_rescheduling_tries=${OCF_RESKEY_rescheduling_tries_default}}

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bogdan Dobrelya (<email address hidden>) on branch: master
Review: https://review.openstack.org/125085

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bogdan Dobrelya (<email address hidden>) on branch: stable/5.0
Review: https://review.openstack.org/125086

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/125085
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=44c1dbbf440f2e8878ffd4bd2589691be49f2e4a
Submitter: Jenkins
Branch: master

commit 44c1dbbf440f2e8878ffd4bd2589691be49f2e4a
Author: Bogdan Dobrelya <email address hidden>
Date: Tue Sep 30 16:52:59 2014 +0200

    Fix dump_cib for corosync provider

    W/o this patch, dump_cib sometimes may
    return an empty xml

    Related-bug: #1375842

    Change-Id: Ia2dafe173f03bce82757d6e9522b405fb0969fb4
    Signed-off-by: Bogdan Dobrelya <email address hidden>