Deployment fails with: pacemaker resource p_vrouter could not be started

[Evgeny Sboychakov]

Steps to reproduce:
1. Create cluster
2. Left the default settings on the Storage Backends tab
3. Add 1 Controller
4. Add 3 Computes
5. No bonding. No DVR. All other settings are default.
6. Deploy the cluster

Expected result:
Deploy passed.

Actual result:
Deploy failed.

[Ilya Kutukov (ikutukov)]

2015-12-14 19:16:23 INFO [96] Casting message to Nailgun:
{"method"=>"deploy_resp",
 "args"=>
  {"task_uuid"=>"b3b625a6-0520-442c-b890-f3e72143e05c",
   "status"=>"error",
   "error"=>
    "Method granular_deploy. Deployment failed on nodes 1.\nInspect Astute logs for the details"}}

Node 1 puppet log

2015-12-14 19:16:21 +0000 Puppet (debug): Executing '/usr/sbin/crm_attribute -q --type crm_config --query --name dc-version'
2015-12-14 19:16:21 +0000 Puppet (debug):
-> Simple primitive 'sysinfo_node-1.domain.tld' global status: stop
   node-1.domain.tld: stop
-> Cloned primitive 'clone_p_vrouter' global status: stop
   node-1.domain.tld: stop
2015-12-14 19:16:21 +0000 Puppet (err): Execution timeout after 600 seconds!
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'
/etc/puppet/modules/pacemaker/lib/puppet/provider/pacemaker_common.rb:706:in `retry_block_until_true'
/etc/puppet/modules/pacemaker/lib/puppet/provider/pacemaker_common.rb:742:in `wait_for_start'
/etc/puppet/modules/pacemaker/lib/puppet/provider/service/pacemaker.rb:111:in `start'
/usr/lib/ruby/vendor_ruby/puppet/type/service.rb:83:in `block (3 levels) in <module:Puppet>'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:197:in `call_valuemethod'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:498:in `set'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:581:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/type/service.rb:94:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:204:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:128:in `sync_if_needed'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:81:in `perform_changes'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:20:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:204:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:217:in `eval_resource'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:in `call'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:335:in `block in thinmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:334:in `thinmark'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:138:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:in `block in apply'
/usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
/usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:in `as_logging_destination'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:168:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:in `block in apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:227:in `run_internal'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:in `block in ...

[Kyrylo Galanov (kgalanov)]

Resource p_vrouter could not be started

Puppet log: http://paste.openstack.org/show/482050/

--
VERSION:
  feature_groups:
    - mirantis
    - experimental
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "291"
  build_id: "291"
  fuel-nailgun_sha: "b6dc9fe8b0e046dc116f1394e9a58887bdf6e91a"
  python-fuelclient_sha: "1d61ed2dbf2183f6c0f9dcd00a4a9ca23aeafaa5"
  fuel-agent_sha: "2f18b7596bc7da79d2f28c34f42620b2090d8a35"
  fuel-nailgun-agent_sha: "a33a58d378c117c0f509b0e7badc6f0910364154"
  astute_sha: "583d68b1cbac03a4f9b6bb9f15921fbbb38159ee"
  fuel-library_sha: "8f2f65f2ed17309d15575501b3d0d7edc41cfe60"
  fuel-ostf_sha: "e77627b0649c597307daa4e27e701ffa230308f0"
  fuel-mirror_sha: "31b9df814960ec69b644ca9b689dacec0c7e10a1"
  fuelmenu_sha: "680b720291ff577f4c058cee25f85e563c96312e"
  shotgun_sha: "a0bd06508067935f2ae9be2523ed0d1717b995ce"
  network-checker_sha: "a3534f8885246afb15609c54f91d3b23d599a5b1"
  fuel-upgrade_sha: "1e894e26d4e1423a9b0d66abd6a79505f4175ff6"
  fuelmain_sha: "7702258023b34e67e94594aef0d679a67fd844c5"

[Bogdan Dobrelya (bogdando)]

Logs snipped http://pastebin.com/DhYB4PTJ
It seems crmd warning: warning: qb_ipcs_event_sendv: new_event_notification (3444-7064-15): Broken pipe (32)
was a RC of the v_router rendered stopped. But it is not clear why it was not restarted either by pacemaker in 10 minutes.

Anyway, this looks like floating issue, lowering to high

[Dmitry Bilunov (dbilunov)]

This bug could be possibly related with #1528488

[Maksim Malchuk (mmalchuk)]

Didn't reproduced on 8.0 iso #374 and 9.0 #35

[Evgeny Sboychakov (ysboychakov)]

Reproduced on 8.0 RC2

[Evgeny Sboychakov (ysboychakov)]

Snapshot attached