clone_p_vrouter global status: stop on deployment

[Stanislaw Bogatkin]

api: '1.0'
astute_sha: c1793f982fda7e3fc7b937ccaa613c649be6a144
auth_required: true
build_id: 2015-04-28_03-40-24
build_number: '358'
feature_groups:
- mirantis
fuel-library_sha: 0e5b82d24853304befb22145ac4aaf3545d295e1
fuel-ostf_sha: b38602c841deaa03ddffc95c02f319360462cbe3
fuelmain_sha: 1ec588d364b9b97f124f6d602dbcc4aa13327218
nailgun_sha: 5e52637d9944c2f4170012560d15ecf89a691af6
openstack_version: 2014.2.2-6.1
production: docker
python-fuelclient_sha: 8cd6cf575d3c101dee1032abb6877dfa8487e077
release: '6.1'
release_versions:
  2014.2.2-6.1:
    VERSION:
      api: '1.0'
      astute_sha: c1793f982fda7e3fc7b937ccaa613c649be6a144
      build_id: 2015-04-28_03-40-24
      build_number: '358'
      feature_groups:
      - mirantis
      fuel-library_sha: 0e5b82d24853304befb22145ac4aaf3545d295e1
      fuel-ostf_sha: b38602c841deaa03ddffc95c02f319360462cbe3
      fuelmain_sha: 1ec588d364b9b97f124f6d602dbcc4aa13327218
      nailgun_sha: 5e52637d9944c2f4170012560d15ecf89a691af6
      openstack_version: 2014.2.2-6.1
      production: docker
      python-fuelclient_sha: 8cd6cf575d3c101dee1032abb6877dfa8487e077
      release: '6.1'

Controller, 2 cinder+compute, Neutron+GRE

Deployment failed with:
2015-04-28 05:55:33 +0000 Puppet (err): Execution timeout after 600 seconds!
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:97:in `fail'
/etc/puppet/modules/pacemaker/lib/puppet/provider/pacemaker_common.rb:641:in `retry_block_until_true'
/etc/puppet/modules/pacemaker/lib/puppet/provider/pacemaker_common.rb:677:in `wait_for_start'
/etc/puppet/modules/pacemaker/lib/puppet/provider/service/pacemaker.rb:110:in `start'
/usr/lib/ruby/vendor_ruby/puppet/type/service.rb:78:in `block (3 levels) in <module:Puppet>'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:197:in `call_valuemethod'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:498:in `set'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:581:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/type/service.rb:89:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:191:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:128:in `sync_if_needed'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:81:in `perform_changes'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:20:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:174:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:187:in `eval_resource'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `call'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:327:in `block in thinmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:326:in `thinmark'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:108:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:164:in `block in apply'
/usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
/usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:108:in `as_logging_destination'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:163:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:125:in `block in apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:124:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:192:in `run'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:268:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:218:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:146:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:470:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:478:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:91:in `execute'
/usr/bin/puppet:4:in `<main>'
2015-04-28 05:55:33 +0000 /Stage[main]/Cluster::Ntp_ocf/Service[p_ntp]/ensure (err): change from stopped to running failed: Execution timeout after 600 seconds!

Seems that problem is happened due to:
root@node-2:~# pcs status
.........
 Clone Set: clone_p_vrouter [p_vrouter]
     Stopped: [ node-2.test.domain.local ]
.........

[Anastasia Palkina (apalkina)]

Cannot reproduce on ISO #357 on VBox

"build_id": "2015-04-27_22-54-38", "build_number": "357", "release_versions": {"2014.2.2-6.1": {"VERSION": {"build_id": "2015-04-27_22-54-38", "build_number": "357", "api": "1.0", "fuel-library_sha": "0e5b82d24853304befb22145ac4aaf3545d295e1", "nailgun_sha": "5e52637d9944c2f4170012560d15ecf89a691af6", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "8cd6cf575d3c101dee1032abb6877dfa8487e077", "astute_sha": "c1793f982fda7e3fc7b937ccaa613c649be6a144", "fuel-ostf_sha": "b38602c841deaa03ddffc95c02f319360462cbe3", "release": "6.1", "fuelmain_sha": "1ec588d364b9b97f124f6d602dbcc4aa13327218"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "0e5b82d24853304befb22145ac4aaf3545d295e1", "nailgun_sha": "5e52637d9944c2f4170012560d15ecf89a691af6", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "8cd6cf575d3c101dee1032abb6877dfa8487e077", "astute_sha": "c1793f982fda7e3fc7b937ccaa613c649be6a144", "fuel-ostf_sha": "b38602c841deaa03ddffc95c02f319360462cbe3", "release": "6.1", "fuelmain_sha": "1ec588d364b9b97f124f6d602dbcc4aa13327218"

root@node-3:~# pcs status
Cluster name:
Last updated: Tue Apr 28 15:42:14 2015
Last change: Tue Apr 28 15:40:39 2015 via crmd on node-3.domain.tld
Stack: corosync
Current DC: node-3.domain.tld (1) - partition with quorum
Version: 1.1.10-42f2063
1 Nodes configured
17 Resources configured

Online: [ node-3.domain.tld ]

Full list of resources:

 Clone Set: clone_p_vrouter [p_vrouter]
     Started: [ node-3.domain.tld ]
 vip__management (ocf::fuel:ns_IPaddr2): Started node-3.domain.tld
 vip__public_vrouter (ocf::fuel:ns_IPaddr2): Started node-3.domain.tld
 vip__management_vrouter (ocf::fuel:ns_IPaddr2): Started node-3.domain.tld
 vip__public (ocf::fuel:ns_IPaddr2): Started node-3.domain.tld
 Master/Slave Set: master_p_conntrackd [p_conntrackd]
     Masters: [ node-3.domain.tld ]
 Clone Set: clone_p_haproxy [p_haproxy]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_dns [p_dns]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_mysql [p_mysql]
     Started: [ node-3.domain.tld ]
 Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server]
     Masters: [ node-3.domain.tld ]
 Clone Set: clone_p_heat-engine [p_heat-engine]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_p_ntp [p_ntp]
     Started: [ node-3.domain.tld ]
 Clone Set: clone_ping_vip__public [ping_vip__public]
     Started: [ node-3.domain.tld ]

Failed actions:
    p_heat-engine_monitor_20000 (node=node-3.domain.tld, call=979, rc=7, status=complete, last-rc-change=Tue Apr 28 15:41:56 2015
, queued=0ms, exec=0ms
): not running

PCSD Status:
  19...

[Oleksiy Molchanov (omolchanov)]

Closing as incomplete, please reopen if reproduced and provide with env asap

[Bogdan Dobrelya (bogdando)]

The same issue with the start went off ahead of a new constraint added http://paste.openstack.org/show/Te9EIYekqOnUB0oQuvnt/